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H.  OBJECTIVES  k  SIGNIFICANCE 


A.  General 

We  continue  to  develop  new  methods  for  recording  and  analyzing  the  electrical 
activity  from  the  brain  of  people  performing  simple  cognitive  tasks.  Our  goal  is  to 
accurately  measure  the  distributed  functional  neural  networks  responsible  for 
human  goal-directed  behaviors. 

Our  short-term  objective  is  to  predict  decrements  in  performance  consequent  to 
attentional  lapses  or  operational  fatigue.  Our  long-term  objective  is  to  develop  an 
instrument  to  non-invasively  measure  the  functional  neuroanatomy  of  the  human 
brain  with  high  temporal  resolution.  Such  an  instrument  would  be  invaluable  for 
neuropsychological  studies  of  cognition,  including  assessment  and  training  of 
cognitive  skills,  for  clinical  studies  such  as  monitoring  recovery  of  function  following 
head  injury,  and  for  on-line  monitoring  of  attention  such  as  detecting  transient 
cognitive  impairments  or  G-induced  loss  of  consciousness  in  the  cockpit. 


B.  Specific 

Recordings  with  up  to  64  scalp  EEG  channels  during  highly  controlled  tasks  are 
now  routine  in  our  laboratory,  as  is  the  sequence  of  signal  processing  operations 
required  to  extract  neurocognitive  information  from  sets  of  single-trial  data 
comprising  up  to  several  billion  bytes.  More  robust  measures  of  the  "functional 
interdependency*  between  electrodes  (event-related  covariance  —  ERG)  have  been 
developed  and  applied  to  data  from  a  bimanual  visuomotor  task  recorded  several 
years  ago.  The  results  of  that  experiment  were  clear-cut  and  consistent  with 
neuropsychological  models  of  the  cortical  activity  accompanying  expectancy, 
stimulus  registration  and  feature  extraction,  movement  preparation  and  execution, 
and  feedback  (see  Section  IV. A).  Many  of  these  results,  along  with  pilot  analyses  of 
intracerebral  recordings  using  a.  primate  model,  cannot  be  explained  by  single 
equivalent-current  dipole  source  models,  but  could  be  understood  in  the  context  of 
multiple,  simultaneously  active  neuronal  sources.  Determining  multiple  neuronal 
sources  associated  with  cognitive  processing  remains  a  formidable  problem,  however, 
since  the  projected  patterns  of  scalp  potential  from  those  sources  overlap  both  in 
time  and  in  space,  and  are  usually  smaller  than  unrelated  spontaneously-generated 
brain  potentials. 

Since  the  main,  unresolved  issue  is  the  origin  of  ERC  patterns  measured  at  the 
scalp,  we  are  now  concentrating  on  improving  our  ability  to  relate  scalp  recordings 
to  source  generator  configurations.  With  improved  technologies  currently  available, 
we  believe  that  several  simultaneous  sources  can  be  resolved. 
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HI.  METHODOLOGICAL  ADVANCES 


A.  Overview  of  Neurocognitive  Pattern  (NCP)  Analysis 
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Thus  far,  there  have  been  four  generations  of  NCP  Analysis.  The  first  measured 
background  EEG  spectral  intensities  while  people  performed  complex  tasks,  such  as 
arithmetic  problems  lasting  up  to  one  minute.  Complex  perceptuo-motor  and 
cognitive  activities  like  these  produce  unique,  spatially  differentiated  scalp  EEG 
spectral  patterns.  These  patterns  had  sufficient  specificity  to  identify  the  type  of 
tasks  being  performed  [1-3].  These  results  agreed  with  previous  reports  of 
hemispheric  lateralization  of  'spatial*  and  'linguistic'  processing.  But  when  the 
tasks  were  controlled  for  stimulus,  response  and  performance-related  factors,  they 
had  identical,  spatially  diffuse  EEG  spectral  scalp  distributions  [4]  (Fig.  l).  Since  no 
patterns  of  hemispheric  lateralization  were  found,  this  suggested  that  previous  and 
most  current  reports  of  EEG  hemispheric  lateralization  may  have  confounded 
electrical  activity  related  to  limb  and  eye  movements  and  arousal  with  those  of 
mental  activity. 
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Results  of  experiments  designed  to  assess  EEG  correlates  of  complex  cognitive  functions.  The  graphs  display  means  over  all 
subjects  of  standard  Korea  of  EEG  spectral  intensities  (expressed  as  changes  from  visual  fixation  values  for  clarity  of 
display)  recorded  during  performance  of  two  tasks  in  Experiment  1  and  three  tasks  in  Experiment  2.  Upper,  middle  and 
lower  sets  of  gTtphs  are  for  spectral  Intensities  in  the  theta,  alpha,  and  beta  bands.  The  abscissae  show  scalp  electrode 
placements:  F3,  left  frontal;  F4,  right  frontal;  C3,  left  central;  C4,  right  central;  P3,  left  parietal;  P4,  right  parietal;  01,  left 
occipital;  and  02,  right  occipital.  Standard  deviations,  which  differed  only  slightly  between  electrode  placements,  are 
indicated  at  the  right  of  each  graph.  (Left)  Tasks  of  Experiment  1  were  one  minute  long  and  involved  limb  movements  and 
uncontrolled  differences  in  stimulus  characteristics  and  performance-related  factors.  (Right)  Tasks  of  Experiment  2  were 
less  than  16  seconds  long  and  required  no  motion  of  the  limbs;  stimulus  characteristics  and  performance-related  factors  were 
also  relatively  controlled.  Although  there  are  prominent  EEG  differences  between  the  uncontrolled  tasks  of  Experiment  1, 
EEG  differences  between  the  relatively  controlled  tasks  of  Experiment  2  are  lacking.  Each  of  the  controlled  tasks  is, 
however,  associated  with  a  remarkably  similar  bilateral  reduction  in  alpha  and  beta  spectral  intensity  over  occipital,  parietal, 
and  central  regions  (3|. 


The  second  generation  of  NCP  Analysis  measured  crosscorrelations  between  91 
pairwise  combinations  of  15  electrodes  recorded  during  performance  of  simple  tasks. 
These  split-second  tasks,  controlled  so  that  only  the  type  of  judgment  varied,  were 
associated  with  complex,  rapidly-shifting  event-related  correlation  patterns  [5]  (Fig. 
2).  By  extracting  differences  between  similar  spatial  tasks,  rapidly  shifting  focal 
patterns  were  found  that  were  in  agreement  with  the  known  functional 
neuroanatomy  of  the  cerebral  cortex  [6,7]  (Fig.  3).  From  these  results,  it  was  clear 
that  it  was  possible  to  measure  salient  aspects  of  the  rapidly  shifting,  complex  mass 
neural  processes  that  are  associated  with  the  successive  information  processing 
stages  of  simple  visuomotor  judgment  tasks. 

The  third  generation  involved  expansion  of  the  recording  and  analysis  systems  to 
operate  on  up  to  64  channels.  The  task  protocol  was  developed  to  allow  a 
controlled  sequence  of  stimuli  in  which  a  person  prepared  for,  and  executed, 
perceptual  judgment  and  motor  response  tasks,  and  received  performance  feedback. 

The  fourth  and  latest  version  of  NCP  Analysis  (ADIEEG-IV)  includes  many 
significant  operational  improvements  (Fig.  4)  [8].  It  has  a  greatly  expanded 
capacity  and  its  operation  is  being  automated  by  development  of  an  AI  expert 
system.  It  is  designed  for  up  to  256  channels,  and  is  implemented  on  a  32-bit 
multiprocessor  system  which  currently  has  3  computing  nodes,  a  6  million  floating- 
point-operations-per-second  capability  and  a  3500  megabyte  on-line  disk  capacity. 

The  data  acquisition  system  is  capable  of  sampling  up  to  256  channels  at  up  to  2 
kHz  sampling  rates  per  channel.  (Current  amplification  capabilities  are  128 
channels.)  Trial  presentation  is  automatically  delayed  until  eye  blinks,  amplifier 
settling  from  eye  blinks,  and  gross  body  movements  have  all  ended.  Up  to  70  EEG 
and  non-EEG  channels  are  monitored  at  a  time  cn  a  color  graphics  screen  to  detect 
electrode  problems.  Other  channels  are  bank-switchable  to  the  monitor.  Averages 
can  also  be  viewed  on-line  to  check  for  event-registered  artifacts. 

Off-line,  an  interactive  graphics  trial  editing  program  is  used  by  an  operator  to 
check  the  decisions  of  the  automatic  on-line  artifact  detectors.  In  practice,  editors 
have  become  sufficiently  skilled  to  check  the  single  trials  of  an  experiment  in 
roughly  the  time  that  it  takes  to  run  the  experiment.  (This  represents  a  speed-up 
of  10-15  times  over  previous  manual  procedures.) 

A  Laplacian  spatial  filter  can  be  applied  using  precise  measurements  of  electrode 
position  to  remove  the  effect  of  choice  of  reference  channel  and  reduce  spatial  blur 
[10].  Additionally,  a  pathway  has  been  built  for  the  inclusion  of  detailed  structural 
information  via  reconstruction  of  actual  scalp,  skull,  and  brain  geometries  using 
Magnetic  Resonance  (MR)  images  (Fig.  5).  This  information  is  used  to  deblur 
average  event-related  potential  waveforms  to  estimated  currents  leaving  the  brain 
(actually,  a  smooth  hypothetical  surface  surrounding  the  brain). 
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Spitlotcmporal  brain  potential  differences  between  two  split-second  tasks,  one  requiring  a  spatial  judgment  and  the  other 
requiring  a  numeric  judgment,  are  *hown  in  each  of  six  interval*  from  juit  before  the  itimuiu*  to  jutt  before  the  response,  A 
pressure  of  the  right  index  finger  was  the  required  response  la  both  ca*es.  The  most  significantly  differing  electrode*  are 
shown  by  circles  (connected  when  grouped  in  pairs),  their  significance  level  by  stippling  of  the  circles,  and  their  most 
prominent  correlations  with  other  electrodes  by  inter-electrode  line*,  A  solid  line  between  two  electrodes  indicates  that  the 
correlation  between  the  electrodes  was  higher  la  the  spatial  task,  while  a  dotted  line  indicate*  higher  numeric  task 
correlations. 

Note  the  contrasts  between  the  spatial  and  numeric  tasks  in  the  preatimulus  interval,  a  finding  that  might  be  interpreted  as 
evidence  of  a  'preparatory  set,'  Also,  the  lack  of  contrast  in  the  preresponse  interval  may  be  interpreted  aa  the  completion 
of  task-specific  perceptual  processing  [6], 


Figure  S 

Spattotemporal  brain  potential  difference*  occurring  between  two  split-second  spatial  tasks  in  each  of  four  intervals  from  just 
before  the  stimulus  to  just  before  the  response.  The  'Move*  task  required  a  pressure  of  the  right  index  finger,  while  the 
*No-Move*  task  required  withholding  the  response.  The  site  of  th  ’.  circle  in  each  interval  is  proportional  to  the  significance 
of  between-taak  difference*  at  the  moat  significant  electrode  in  that  interval  (i.e.,  Fr<.01,  PzC.001,  P«, 00005,  C3C.000005). 
A  solid  line  between  two  electrode*  Indicates  that  the  correlation  was  higher  in  the  *Move'  task,  while  a  dotted  line 
indicates  higher  *No-Move*  task  correlations.  The  midline  frontal  (Pi)  electrode  focus  in  the  prestimuius  interval  suggests  a 
differential  preparatory  net;  the  midline  parietal  (Pi)  focus  la  the  Interval  ceotered  os  the  N1  and  P2  evoked  potential  peaks 
suggests  task-specific  feature  extraction  and  pattern  recognition;  the  right  parietal  (P4)  focus  In  (he  interval  centered  on  the 
PI*  evoked  potential  peak  suggests  differential  spatial  judgments  between  the  *Move*  and  the  'No-Move'  tasks,  while  the 
left-centered  (C5)  focus  in  the  pre-response  Interval  suggests  preparation  of  the  right  index  finger  response  in  the  'Move' 
task  (6,7). 
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Figure  4 

The  ADIEEG-IV  system  for  quintification  of  event- related  brain  signals  If  shown.  Separate  subsystems  perform  on-line 
experimental  control  and  data  collection,  data  selection  and  evaluation,  signal  processing  and  pattern  recognition.  Current 
capacity  Is  128  channels.  Spherical-head  spatial  debarring  modules  have  been  implemented,  and  multiple  source  modeling 
algorithms  are  being  developed.  Digital  tapes  of  magnetic  resonance  Images  or  of  eleetrophyiiologieal  data  from  other 
laboratories  are  converted  Into  the  ADIEEG  data  format  using  gateway  programs.  The  data  are  then  processed  using  the 
same  program  modules  as  data  collected  in  the  EEG  Systems  Laboratory  |0|. 
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M*cnetle  Retooance  of  mld-ufltUl  lection  of  •  •ubject'i  cranium.  EEG  electrode*  are  luperimpoaed  [12). 


The  resulting  sets  of  purified  and  selected  EEG  (or  MEG,  for  imported  data) 
trials  are  then  entered  into  one  of  a  number  of  possible  single-trial  or  average 
event-related  potential  analyses.  Averaging,  filtering,  time-varying  spectral 
analysis,  modeling,  ERC  measures,  and  multivariate  pattern  recognition  are  all 
supported  under  this  system.  Finally,  two-  and  three-dimensional  perspective  plots 
are  available  to  view  the  results  of  these  analyses. 

This  system  is  currently  being  improved  to  provide  greater  ease  of  operation. 
While  recording,  channels  that  are  not  currently  on-screen  will  nonetheless  be 
automatically  monitored  for  serious  problems  such  as  flattening  or  saturation. 
Additionally,  other  artifacts  such  as  eye  movement  and  muscle  contamination  will 
be  automatically  marked  by  on-line  pattern  classification  programs  (Fig.  6)  [11]. 
Also,  algorithms  are  under  development  to  recovei  trials  with  non-saturating 
artifacts.  Eye  blinks  or  eye  movements,  for  example,  will  be  removed  using  least 
squares  noise  cancellation  given  EOG  reference  electrodes  near  the  eyes.  The 
system  will  then  automatically  calculate  the  optimum  (largest)  subset  of  trials  that 
behaviorally  balance  for  a  requested  set  of  variables  on  each  split  of  the  data 
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required  to  test  an  experimental  hypothesis  (e.g.,  between  low  and  high  accuracy 
trials).  This  will  be  followed  by  a  manual  check  of  the  split  distributions,  along 
with  corrections  in  exceptional  cases.  This  will  be  speeded  up  by  a  convenient, 
window-oriented  user  interface.  Further  development  of  the  analysis  system  will 
allow  each  step  to  be  requested  by  the  user  without  reference  to  file  names  or 
directory  locations. 


CLASSIFIER -DIRECTED  ARTIFACT  DETECTION 


figure  6 

Claaslfier-dlreeted  artifact  rejection.  A  system  consisting  of  five  parallel  detectors  Is  used  to  find  contaminants.  Three  of  the 
detectors  incorporate  layered-neural- network  classifiers  to  chooae  and  weight  feature  combinations.  In  this  way,  a  precise 
automated  procedure  replaces  sole  reliance  on  ad  hue  waveform  detectors  and  subjectively  determined  thresholds  [11). 
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B.  Recent  Developments  in  NCP  Analysis 
1.  Data  Collection 

a.  64-channel  EEG  Recording  Technique  and  Automated  Artifact  Rejection. 
Recording  capacity  has  been  expanded  to  a  full  64  EEG  channel  capacity  to  provide 
uniform  scalp  coverage  with  an  interelectrode  distance  of  about  3.25  cm  on  an  adult 
head  (Fig.  7).  All  recordings  are  referenced  to  one  mastoid  (usually  M2);  the  other 
(Ml)  is  recorded  as  a  separate  channel  so  that  a  laterally  balanced  (M1+M2) 
reference  may  be  computed  off-line,  avoiding  the  undesirable  boundary  condition  on 
the  potential  field  created  by  linked-ear  references.  Signals  are  digitized  at  12  bits 
full  scale  at  256  samples/sec,  and  stored  on  magnetic  tape.  Up  to  70  channels  are 
monitored  at  a  time  for  electrode  problems.  Improved  automated  artifact  rejection 
algorithms  uting  neural-network  pattern  recognition  to  combine  amplitude, 
frequency,  and  topographic  features  [llj  are  currently  applied  off-line,  and  are  being 
converted  to  on-line  analysis.  These  algorithms  satisfactorily  perform  a  preliminary 
data  screening. 


Figure  7 

Expanded  10-20  System  of  electrode  poeltioni.  Additional  rowi  of  electrodes,  interpolated  between  the  International  10-20 
System  coronal  rows,  have  the  letter  a  tor  ‘anterior*  added  to  tbe  designation  for  the  next  row  anterior,  e.g.,  aPr  for 
anterior  parietal  addllne  electrode.  With  64  electrodes,  the  average  distance  between  electrodes  on  an  adult  head  is  ~3.26 
cm  (9). 
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Neural-network  pattern  classification  techniques  were  used  to  evaluate  the 
performance  of  two  techniques  for  eye-movement-artifact  removal:  amplitude 

SSffcS!"  SUlijeCtral  ®ubt.racti,?n  (Fi*  «)•  Although  visual  comparison  showed 
that  both  methods  are  effective,  the  objective  evaluation  technique  suggested  that 

the  spectral  subtraction  method  may  in  some  instances  be  more  effective  for  eve 
bunk  removal.  J 
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b.  Digitization  of  Electrode  Positions.  Before  and  after  a  recording  session,  the 
position  of  the  external  center  of  the  grommet  of  each  electrode  is  measured  in 
three  dimensions  using  a  3-D  digitizer  (Fig.  9).  The  participant  rests  his  head  in  a 
padded  stabilizing  jig  for  the  5  minutes  required  to  digitize  the  positions  of  all 
electrodes.  Correction  to  scalp  positions  is  made  by  a  least-squares  fit  Multiple 
Linear  Regression,  which  yields  the  general  ellipsoid  surface  best  fitted  to  the  set  of 
digitized  positions,  and  the  digitized  coordinates  are  translated  to  a  coordinate 
system  centered  on  this  ellipsoid. 

c.  MR  Imaging  for  Determining  Positions  of  Electrodes  and  Cortical  Structures. 
Full  sets  of  Magnetic  Resonance  Images  (Fig.  10)  are  made  at  UCSF  Long  Hospital 
facility.  These  are  high-  resolution  (0.8  mm  pixel)  3- ax  is  cross-section  images  of  soft 
tissue  5  mm  or  1  cm  apart  over  the  whole  volume  of  the  head.  The  pictures  are 
digitized  to  give  coordinate  surfaces  for  scalp,  outer  and  inner  skull  surfaces,  and 
cerebral  ourface,  including  loci  of  major  fissures.  Comparison  of  electrode  positions 
and  MR  measurements  provides  correlation  of  electrode  positions  and  cortical  areas, 
allowing  comparison  of  functional  neuroanatomy  across  individual  participants  for 
localized  sensory  and  motor  cortical  ERP  components.  Direct  measurement  of  scalp 
and  skull  thicknesses  provides  accurate  information  for  use  in  spatial  deconvolution 
(see  below). 

2.  Initial  Data  Analysis 

a.  Trial  Selection  Using  Pattern  Recognition.  A  great  deal  of  effort  has  been 
expended  in  attempts  to  form  improved  estimates  of  the  averaged  ERP  (reviews  in 
[14-17]).  Most  methods  assume  that  task-related  signals  are  present  in  every  trial 
and  also  have  inherent  assumptions  about  the  statistical  properties  of  signal  and 
noise.  We  have  developed  a  method  for  ERP  estimation  without  the  first 
assumption  and  with  a  relaxed  second  assumption  [18].  To  do  this,  separate 
averages  are  formed  from  trials  with  and  without  detectable  event-related  signals. 
In  the  averages  of  trials  with  detectable  signals,  the  ERP  peaks  are  enhanced  in 
comparison  with  the  original  averages.  The  averages  of  trials  without  detectable 
signals  resemble  the  background  EEG  (Fig.  11). 

Several  dozen  applications  of  this  technique  suggest  that  it  is  successful  in 
identifying  trials  with  a  poor  signal-to-noise  ratio.  The  technique  thus  provides  a 
simple  method  of  ERP  estimation  with  minimal  a  priori  assumptions  about  the 
characteristics  of  the  signal. 

b.  Wtgner  ( Time-Frequency )  Distributions.  The  ERP  waveform  is  a  function  of 
time  and  does  not  provide  explicit  frequency  information.  Power  spectra  of  ERP 
waveforms  provide  frequency  information  but  obscure  time-dependent  phenomena. 
A  view  of  the  spectrum  as  it  changes  over  time  would  give  a  new  view  of  the 
evolution  of  different  frequency  components  of  the  ERP.  A  simple  approach  would 
be  to  compute  the  spectrum  over  highly  overlapped  windows  of  the  average  ERP. 
However,  such  a  'spectrogram'  would  smear  together  events  within  each  analysis 
window.  While  there  is  an  unavoidable  trade-off  between  frequency  and  time 
resolution  (known  as  the  Heisenberg  Uncertainty  Principle),  the  spectrogram  is  an 
inflexible  way  of  determining  this  choice. 
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Us«  of  sUUstlcsJ  psUcra  recognition  utlyti*  to  remove  trills  without  detectable  task-related  signals  from  a  set  of  single- 
trial  ERPs  (actually  event- related  current  source  densities).  This  results  In  an  average  ERP  with  a  higher  signal-to-nolsc 
ratio  obtained  from  }««  trials.  Unlike  optimal  filtering  methods,  s  priori  hypothesised  models  of  the  structure  and 
statistical  properties  of  signal  and  noise  components  are  not  required.  (From  (17,18|.)  (a)  An  original  average  ERP  formed 
from  100  presentations  of  a  visual  numeric  stimulus,  (b)  Average  of  U1  trials  with  consistent  task-related  signals  in  the  P2 
Interval.  Trials  were  selected  from  the  original  set  of  199  by  applying  a  pattern  recognition  algorithm  to  distinguish  a  125- 
ms,  P2  time  series  segment  from  a  pre-cue  'baseline*  segment,  Note  the  greatly  Increased  sire  of  the  event-related  peaks 
and  lower  frequency  wave  forms,  (c)  Average  of  68  trials  which  did  not  have  consistent  task-related  signals  In  the  P2 
Interval.  Note  the  relative  lack  of  event-related  activity  and  the  dominance  of  ongoing  EEG  alpha  waves. 


A  preferred  method  is  to  compute  a  general  function  of  time  and  frequency  called 
the  Wigner  Distribution  [19-21]  which  approximates  the  instantaneous  energy  for  a 
given  time  and  frequency,  hi  practice,  ERPs  often  show  strong  enough  energy 
'peaks’  in  the  Wigner  Distribution  to  validate  very  simple  interpretations  of  the 
time  and  frequency  locations  of  signal  energy  (Fig.  12).  The  Wigner  Distributions 
are  being  used  to  determine  digital  filter  characteristics  which  produce  optimal 
time-frequency  resolution  for  a  given  data  set. 
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Figure  13* 

Figure  U  gives  two  representations  of  eight  average  ERP  channels  (or  'move'  aad  *no-move'  cognitive  tasks.  The  view  is 
of  the  top  of  the  head,  with  the  nose  at  the  top  of  each  set  of  eight  chaaoels. 

Mgure  12a  shows  the  average  time  series  of  40  no-move  and  97  move  trials.  These  trials  were  selected  by  a  statistical 
patters  recognition  algorithm  as  having  consistent  task-related  signals  that  differed  between  the  move  asd  nomove  tasks. 
Three  of  the  most  commonly  studied  ERP  peaks  —  Nl,  P2,  P3  —  are  Indicated  on  the  Pi  channel  of  the  no-move  task.  Of 
these,  the  P3  peak  is  larger  in  the  infrequently  occurring  no-move  trials. 
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Figure  13b 

The  pseudo-Wigner  distribution  of  the  analytic  signal  of  the  same  data.  This  representation  shows  that  the  event-related 
processes  are  changing  rapidly  in  both  time  and  frequency.  The  i.rst  moment  along  the  time  axil  for  each  frequency  is  the 
(roup  delay,  while  the  tint  moment  along  the  frequency  axis  is  the  Instantaneous  frequency.  There  Is  a  buildup  in  energy 
after  the  stimulus,  and  a  general  increase  in  frequency  until  the  energy  concentration  between  the  time  of  the  Nl  and  P2 
peaks  begins  to  fall  off  (moat  prominent  in  the  Pa  channel).  Then  there  is  a  glide  down  in  frequency  in  the  no-move  task 
(moat  prominent  in  the  Pa  and  aOt  channels),  which  culminates  In  a  concentration  of  energy  around  the  time  of  the  P3 
peak.  (From  |19|.) 
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c.  Balancing  Data  Sets.  Special  attention  is  given  to  the  preparation  of  data 
sets,  since  we  believe  that  this  is  a  highly  neglected  aspect  of  experimental  control. 
In  order  to  form  balanced  pairs  of  data  sets  for  each  neurophysiological  hypothesis, 
the  total  set  of  artifact-free  trials  from  each  recording  is  submitted  to  an  interactive 
program  that  displays  the  means,  t-tests  and  histogram  distributions  of  about  50 
behavioral  and  other  variables  [7].  Pairs  of  data  sets  can  be  quickly  inspected  for 
significant  differences  in  variables  not  related  to  the  hypothesis  and  pruned  of 
outlier  trials  until  balanced,  usually  to  a  conservative  alpha  of  .2.  The  program 
greatly  facilitates  the  handling  of  large  amounts  of  data  and  permits  a  high  degree 
of  control  of  stimulus,  behavioral  and  other  variables  which  would  not  otherwise  be 
possible.  It  outputs  labeled  lists  of  trials  to  be  submitted  to  the  signal  processing 
and  analysis  programs  of  our  system  that  vary  only  according  to  the  chosen 
hypothesis.  Variables  include  stimulus  parameters,  responding  finger,  response 
onset  and  inter-movement  times,  response  movement  force,  velocity,  acceleration 
and  duration,  error  and  adaptive  performance  measure.  There  are  also  indices  of  eye 
movement  and  EMG  activity  computed  in  separate  500-msec  epochs  before  and 
after  the  onset  of  cue,  stimulus,  response  and  feedback. 

d.  EEG  Spatial  Signal  Enhancement.  Attempts  to  study  the  information 
processing  activity  of  the  brain  using  scalp-recorded  ERPs  have  suffered  from  the 
fact  that  potentials  induced  at  the  scalp  by  volume  conduction  from  cerebral 
sources  are  subject  to  considerable  distortion  by  the  intervening  media  of  cerebrum, 
cerebro-spinal  fluid,  skull,  and  scalp.  The  result  is  a  smearing  or  spatial  low-pass 
filtering  of  the  pattern  of  activity  in  the  brain.  This  section  reviews  two  spatial 
filters  that  may  increase  the  resolution  of  the  recorded  topographical  map  of  brain 
activity:  Laplacian  Derivation  and  Deconvolution.  Each  provides  a  measure  of  the 
local  density  of  normal  current  at  the  brain-skull  surface  due  to  the  activity  of 
neuronal  sources  within  the  brain.  Each  is  independent  of  the  reference  electrode 
and  applicable  to  single  trials  as  well  as  to  averages.  Time  series  analysis  and 
hypothesis  testing  may  be  performed  on  the  filtered  densities  in  identical  fashion  to 
potential  maps. 

The  Laplacian  Derivation  (LD)  [22-25],  an  estimate  of  the  two-dimensional 
(tangential)  Laplacian  operator  at  each  scalp  position,  is  proportional  to  the  local 
density  of  current  flow  through  the  skull.  Its  application  has  a  localizing  effect  since 
it  is  a  differential  property  of  the  potential  field. 

The  other  filter  models  the  passive  tissues  of  skull  and  scalp  in  order  to 
deconvolve  the  measured  scalp  potential  distribution  into  a  topographical  map  of 
outgoing  current  density  just  above  the  surface  of  the  cortex.  This  sheet  of 
"dipoles"  is  equivalent  to  a  "Helmholtz  double  layer"  that  completely  characterizes 
the  activities  of  the  real  sources  within  [26].  The  surface  of  the  cortex  is  chosen 
because  all  the  neuronal  sources  are  inside  it.  It  must  be  stressed,  however,  that 
the  deconvolved  dipole  density  is  not  a  model  of  sources  in  the  sense  that  it  will  not 
"correct"  the  false  impression  of  two  radial  sources  created  by  a  single  tangential 
source.  This  deconvolution  is  similar  in  concept  to  a  'software  lens"  applied  by 
Freeman  [27]  to  recordings  from  the  surface  of  the  olfactory  bulb  in  the  rabbit, 
although  the  same  planar  homogeneous  model  is  not  applicable  to  scalp-recorded 
data.  It  embodies  the  same  principle  as  a  treatment  by  Nicholas  and  DeLoche  [28] 
of  scalp-recorded  ERPs.  Since  source  modeling  [29-31]  requires  an  identical  model 


of  the  tissues  outside  the  brain,  this  computationally  cheap  deconvolved  current 
density  may  serve  as  a  starting  point  for  source  modeling,  with  the  effects  of  the 
passive  tissue  already  removed. 

Here  we  compare  common  average  reference,  Laplacian  Derivation,  and 
deconvolution  procedures.  The  interchannel  covariances  for  the  three  derivations 
are  calculated  on  a  visual  ERP,  as  well  as  by  a  linear  regression  of  interchannel 
covariance  against  the  expected  volume  conduction.  Additionally,  simulation 
studies  investigate  the  relative  advantages  of  the  filters  as  a  function  of  electrode 
spacing. 

Potentials  were  recorded  from  a  right-handed  adult  male  performing  a  task 
involving  a  brief,  foveally  presented,  visual,  single-digit  numeric  stimulus  followed 
by  a  precise  motor  response  (see  Section  IV. A).  Fifty  scalp  electrodes  were  placed 
in  an  extension  of  the  standard  International  10-20  System,  with  additional  coronal 
rows  interpolated  between  the  standard  rows  (see  Section  IV.A.l.c).  Recording  was 
done  in  bipolar  chains  of  eight  electrodes;  digitized  values  were  converted  offline  to 
a  common  average  reference  [32].  Electrode  positions  were  measured  using  a 
stereophotogrammetry  system  developed  for  craniofacial  research  and  treatment 
[33,34].  The  Laplacian  Derivation  and  deconvolution  were  implemented  using  the 
measured  electrode  coordinates  to  calculate  the  interelectrode  distances.  All  data 
were  edited  for  artifacts  on  a  channel-by-channel  basis  by  two  independent  experts. 

The  Laplacian  Derivation  estimate  at  each  electrode  was  implemented  as  the 
average  directional  derivative  divided  by  the  average  inverse  distance.  The  latter 
quantity,  however,  may  differ  considerably  in  a  practical  montage.  Worse,  using  this 
expression  for  electrodes  on  the  periphery  of  the  montage  amounts  to  assuming  that 
the  average  directional  derivative  on  the  outside  is  the  same  as  that  on  the  inside, 
which  is  hardly  warranted  in  the  absence  of  information. 

A  simple  three-sphere  head  model  was  used  for  the  deconvolution.  The  electric 
potential  at  each  scalp  position  was  the  linear  superposition  (convolution)  of 
contributions  from  all  sources.  If  z  is  a  vector  expressing  the  scalp  potential 
distribution,  and  x  is  a  vector  of  "dipole*  strengths,  with  H  a  matrix  representing 
the  transmission  between  cortical  "dipples*  and  scalp  potentials,  then  this 
convolution  may  be  written  as 


z  =  H  x 


The  elements  of  the  matrix  H  are  calculated  in  the  context  of  a  particular  model  of 
the  tissues  by  solving  the  boundary  value  problem  governing  the  volume  conduction 
between  brain  and  scalp.  The  deconvolution  was  done  using  a  standard  model  of  the 
head  as  three  concentric  spheres:  brain,  skull,  and  scalp  [24,28-31].  Discrete  current 
dipoles  representing  an  approximation  to  a  continuous  distribution  of  normal 
current  are  posited  at  the  brain-skull  boundary  beneath  each  electrode.  The  values 
for  the  thicknesses  of  skull  and  scalp  and  the  resistivities  of  the  tissues  were  taken 
from  standard  sources  [35-37]. 


Isofunction  contour  plots  for  common  average,  Laplacian  Derivation  (LD),  and 
deconvolution  are  shown  in  Figure  13  for  a  single  time  point  at  an  early  peak  of  the 
averaged  evoked  potential.  There  is  apparently  more  spatial  detail  in  the  Laplacian 
Derivation  and  deconvolution  than  in  the  common  average.  A  similar  result  was 
apparent  at  the  many  different  time  points  that  were  examined. 


In  practice  there  is  noise  n  present  in  the  measured  scalp  potentials  z’,  z’  =  z  + 
n.  The  matrix  H  may  be  singular  or  ill-conditioned,  so  that  the  inversion  may 
require  calculation  of  a  pseudo-inverse  or  its  equivalent  using  an  orthogonal 
transformation.  Use  of  scalp  data  from  multiple  time  points  in  the  vicinity  of  a  peak 
in  the  evoked  response  may  lead  to  a  better  estimate  of  x.  Still,  simple  inversion  of 
the  convolution  presented  here  is  well  conditioned,  with  the  determinant  of  H  equal 
to  15.4,  and  the  condition  number  of  the  inversion  equal  to  34.4,  corresponding  to 
about  5  bits  of  noise.  Householder  inversion  of  4  to  7  points  in  the  vicinity  of  a 
peak  produced  topographical  maps  virtually  identical  to  the  simple  inversion.  A 
simple  inversion  is,  therefore,  considered  adequate  to  the  problem,  given  the  other 
limitations  of  the  model. 

Like  the  Laplacian  Derivation,  the  deconvolution  has  problems  at  the  periphery 
of  the  recording  electrode  array.  The  model  assumes  that  current  outflow  (or 
inflow)  past  the  outer  dipole  positions  makes  a  negligible  contribution  to  the 
potential  at  each  scalp  electrode.  In  this  implementation,  values  of  Laplacian 
Derivation  and  deconvolved  dipole  density  were  treated  with  caution  at  the 
periphery. 

For  each  derivation  (common  average,  Laplacian  Derivation,  and  deconvolution) 
covariances  were  calculated  between  all  pairs  of  non-peripheral  estimates  on 
stimulus-locked  averages  from  0.5  sec  before  the  stimulus  to  1.0  second  after.  Such  a 
covariance  should  primarily  reflect  the  effects  of  volume  conduction,  since  it  spans 
intervals  with  different  functional  topography.  Thus  the  point-spread  function 
should  be  a  good  predictor  of  covariance  for  the  common  average  derivation  and  a 
poor  predictor  for  the  others.  A  linear  regression  was  performed  to  test  this 
hypothesis. 

Simulation  calculations  were  performed  to  determine  the  response  of  each 
derivation  to  a  single  dipole  source  at  a  succession  of  positions.  Using  the  three- 
sphere  model  described  above,  a  single  dipole  was  assumed  at  positions  beginning  at 
the  cortex  below  Cz  and  progressing  forward  along  the  midline  at  64  equal  intervals 
of  polar  angle  for  a  total  of  0.8  radians.  For  each  dipole  position,  the  potential  at 
each  of  the  50  measured  electrode  positions  was  calculated.  For  each  such  scalp 
potential  distribution,  the  Laplacian  Derivation  and  deconvolved  source  strength  at 
Cz  were  calculated,  as  well  as  the  potential  at  Cz. 

In  order  to  investigate  the  effect  of  closer  electrode  or  dipole  spacings,  sets  of 
position  coordinates  were  prepared  from  the  original  measured  set  with  all  polar 
angles  reduced  to  0.75  and  0.5  of  the  originals.  Thus  all  the  interelectrode  spacings 
were  reduced  proportionately.  The  responses  of  the  Laplacian  Derivation  and 
deconvolution  were  calculated  using  these  sets  of  coordinates  for  both  source  and 
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Isofunction  contour  plots  of  a  visual  evoked  potential  10&  msec  after  the  foveal  presentation  of  a  single-digit  number,  (a) 
Sc&lp  potential  (common  average  reference),  (b)  Laplacian  Derivation,  (c)  Dipole  strength  from  deconvolution.  In  (a)  dots 
are  areas  of  positive  potential;  in  (b)  dots  are  exiting  current?;  in  (c)  dots  are  positive  'dipole  strengths.'  In  each  picture, 
each  contour  interval  represents  a  10%  change  from  its  neighbor,  the  maximum  value  being  at  the  back  of  the  bead. 


electrode  positions.  The  responses  were  also  calculated  using  the  original  measured 
coordinates  for  the  electrodes  and  the  0.75-reduced  set  for  the  sources.  Use  of  such 
reduced  sets  for  the  sources  corresponds  to  the  situation  where  one  has  prior 
knowledge  about  the  actual  sources  of  the  potentials,  so  that  the  Helmholtz  layer 
may  be  assumed  to  be  of  approximately  zero  strength  outside  a  restricted  area. 

The  results  of  the  linear  regression  of  covariance  against  pointspread  function  are 
shown  in  Table  1.  As  expected,  the  common  average  covariances  manifested  a  large 
slope  and  a  large  ratio  of  regression  variance  to  residual  variance.  The  Lapiacian 
Derivation  caused  a  considerable  reduction  in  both  these  quantities,  while  the 
deconvolution  effected  even  more  reduction. 


TABLE  1 


INTERELECTRODE  COVARIANCE  (1) 

REGRESSED  VERSUS  POINT  SPREAD  FUNCTION 

SPATIAL  Slops  Intarcapt  r-ratio(2) 

OPERATOR 

2375.49  I  -221.51 

144.19  1  -15.53 

Dsoonvolutlon  |  21.95  -4.18  I  10.6 


(1)  IntmWctrod*  covuiuco  euleuUttd  over  itimultu-loekad  ivnip  from  0.6  ft  before  to  1.0  tec  liter  itlmului. 

(2)  Ratio  of  retreetloa  to  reeidud  variance,  df  m  (1,664). 


The  response  of  the  three  derivations  to  the  progression  of  simulated  source 
positions  is  shown  in  Figure  14  (a,b,  and  c)  for  the  measured  positions,  0.75-  and 
O.&O-reduced  positions,  respectively.  The  point-spread  function  is,  of  course,  the 
same  for  each.  The  resolution  advantage  of  the  deconvolution  increases  as  the 
electrode  spacing  decreases.  Figure  14d  shows  the  deconvolution  response  for 
measured  electrode  positions  and  0.75-reduced  source  positions,  as  well  as  the 
deconvolution  response  from  Figure  14b  for  comparison.  This  shows  that  the 
resolution  of  the  deconvolution  may  be  usefully  improved  if  one  has  knowledge  of 
the  general  location  of  actual  sources  within  the  brain. 


Cofeaon  Avaraga 
Local  Lapiacian 


Simulation  results  showing  response  o t  each  derivation  to  radial  dipole*  ai  a  function  of  tcalp  distance,  In  (a)-(c),  the  solid 
line  U  from  tie  deconvolution,  the  dashed  line  Is  from  the  I,apl*cl*n  Derivation,  and  the  dotted  line  le  from  the  evoked 
potential,  (a)  4  •cm  spacing  of  principal  electrode*  and  dipoles,  (b)  J-cm  spacing  of  electrode*  and  dipole*,  (c)  2*cm  spacing 
of  eltctrode*  md  dipoles,  Note  that  the  Improvement  of  deconvolution  over  local  Laplaclan  Increase*  a*  the  electrode 
spacing  decree ie*.  (d)  Deconvolution  response  with  k-cm  electrode  spacing  (solid)  compared  with  response  calculated  with 
4-cm  electrode  spacing  and  tern  dipole  spacing  (dashed).  Some  approximate  knowledge  of  the  location  of  neuropil  sources 
tan  provide  resolution  equivalent  to  having  closer  spaced  electrodes, 


The  simulation  and  empirical  results  suggest  that  the  covariance  due  to  volume 
conduction  is  somewhat  more  reduced  by  the  deconvolution  than  by  the  Laplacian 
Derivation.  Although  this  seems  to  imply  that  the  deconvolution  improves  the 
resolution  of  brain  activity,  simultaneous  scalp  and  intracerebral  recordings  are 
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required  to  determine  if  this  is  actually  so,  and  many  additional  recordings  need  to 
be  analyzed  to  see  if  these  results  replicate.  In  any  case,  in  the  simulations,  the 
advantage  of  the  deconvolution  over  the  Laplacian  Derivation  increases  as  the 
electrode  spacing  is  decreased.  The  computational  requirement  of  a  matrix 
multiplication  on  each  time  point  for  either  method  is  not  prohibitive  for  use  on 
averages  or  sets  of  single  trials. 

Both  deconvolution  and  source  derivation  are  affected  by  inhomogeneity  in  the 
thicknesses  or  resistivities  of  the  skull  and  scalp.  If  the  skull  is  thinner  or  less 
resistive  in  a  particular  area,  more  current  will  flow  through  it  and  the  value  of 
Laplacian  Derivation  or  deconvolved  activity  will  be  larger  than  the  actual  brain 
activity.  For  either  technique  the  solution  must  involve  a  more  accurate  and 
detailed  modeling  of  the  tissues.  One  may  measure  the  scalp  and  skull  thicknesses 
on  each  individual,  using  magnetic  resonance  images,  ultrasound  probes,  or  other 
methods.  Most  satisfactory  would  be  the  use  of  a  finite-element  or  finite-difference 
calculation  to  model  the  tissues  in  detail,  including  fissures  and  orifices  such  as  the 
orbits. 

e.  MEG  Data  Analysis.  A  collaboration  is  underway  with  the  laboratories  of 
Drs.  Williamson  and  Kaufman.  The  purpose  of  this  effort  is  to  make  a  gateway  for 
their  MEG  data  into  the  ADIEEG-IV  system  so  that  source  modelling  using  both 
EEGa  and  MEGa  can  be  undertaken.  (In  other  research,  we  have  developed 
adaptive  digital  filters  for  removal  of  environmental  noise  from  MEGs.) 

3.  Event-Related  Covariances  (ERCs) 

A  number  of  steps  are  currently  performed  in  computing  ERCs.  Because  of  the 
size  of  the  single-trial  data  sets  (up  to  500  megabytes  for  each  person) ,  a  large  on¬ 
line  disk  capacity  is  required.  As  already  described,  the  first  pass  reduces  spatial 
smearing  and  then  selects  intervals  and  trials  with  task-related  information  to 
enhance  the  sign&l-to-noise  ratio  and  reduce  the  amount  of  data  prior  to  measuring 
ERCs.  The  second  pass  measures  ERCs  on  band-pass-filtered,  enhanced  (optional), 
averages  from  the  reduced  data  set.  This  second  pass  requires  the  following  steps: 

Compute  ERCs:  Select  digital  bandpass  filters  and  intervals  for  measurement  by 
examining  ERPS,  amplitude  distribution  maps  and  Wigner  distributions.  Compute 
multilag  crosscovariance  functions  between  ail  pairwise  channel  combinations  of  the 
enhanced  (optional),  filtered  and  decimated  averages  in  each  selected  analysis 
window.  Use  the  magnitude  of  the  maximum  crosscovariance  function  and  its  lag 
time  as  features  characterizing  the  ERC  [38-41). 

Estimate  significance  of  ERCs:  Estimating  the  significance  of  ERCs  requires  an 
estimate  of  the  standard  deviation  of  the  "noise*  ERC.  It  is  obtained  as  follows:  (1) 
Random  intervals  in  each  single  trial  of  the  ensemble  are  averaged.  (2)  ERC 
analysis  is  performed  on  a  filtered  and  decimated  version  of  the  resulting  "noise" 
averages,  yielding  a  distribution  of  "noise"  ERCs.  (3)  Multiple  comparisons  are 
accounted  for  with  a  Duncan  procedure. 

Graph:  The  most  significant  ERCs  in  each  interval  are  graphed  (Fig.  15). 
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Flgur*  16 

An  Uluatrfttlon  of  a  single  covwUnce  between  the  two  channel)  shown  at  the  right,  Electrode  positions  are  Indicated,  and 
the  rclatlouihlp  of  a  covariance  line  to  the  theta-filtered  averaged  event-related  timeseries  from  the  two  corresponding 
electrode)  U  shown,  In  all  covariance  diagrams,  the  width  of  a  line  Indicates  the  significance  of  covariance  between  2 
electrodes  and  its  color  indicates  the  magnitude  of  time  delay,  When  the  color  of  the  arrow  Is  the  same  as  the  covariance 
line,  the  covariance  is  positive;  when  the  arrow  is  gray,  the  covariance  is  negative.  The  arrow  points  from  the  leading  to  the 
lagging  channel.  (Specific  values  of  significance  and  delay  are  given  on  each  diagram.) 
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Compare  ERC  maps  between  conditions :  Test  the  difference  in  means  of 
significant  ERCs  between  conditions  with  an  ANOVA  and  post-hoc  t-tests.  Measure 
similarity  between  multivariate  ERC  maps  with  an  estimate  of  the  correlation 
between  them.  Calculate  these  estimates  using  a  distribution-independent 
"bootstrap'  Monte  Carlo  procedure  [42],  which  generates  an  ensemble  of  correlation 
values  from  randomly  selected  choices  of  the  repeated  measures.  This  will  also  yield 
a  confidence  interval  for  the  estimates. 
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IV.  RECENTLY  COMPLETED  EXPERIMENTS 


A.  Bimanual  Numeric  Visuomotor  Experiment  [38-40] 

This  study  represented  an  attempt  to  non-invasively  measure  the  functional 
interrelationships  of  the  human  brain  during  simple  goal-directed  behaviors  that 
required  integrated  processing  by  many  brain  regions.  Prior  studies  using 
intracerebral  electrodes  have  suggested  that  there  is  a  consistent  morphology  and 
timing  of  macropotentials  from  functionally  related  brain  regions  [43-45],  Full  four 
sec-long  trials  spanning  prostimulus,  stimulus,  response  execution  and  feedback 
intervals  were  analyzed.  Between-channel  covariance  and  time-delay  measures  were 
derived  from  pairs  of  filtered  Laplacian  Derivation  (LD)  average  waveforms, 
enhanced  by  rejection  of  trials  without  event-related  signals,  in  brief  intervals 
spanning  evoked  potential  components.  Stimulus-  and  response-locked  covariance 
patterns  were  consistent  with  functional  neuroanatomical  models  of  visual  stimulus 
processing  and  motor  execution.  Prestimulus  patterns  predicted  subsequent 
response  accuracy  by  each  hand.  Feedback  patterns  also  distinguished  accurate 
from  inaccurate  performance. 

L  Methods 

a.  Subjects.  Seven  healthy  male  adults  participated  in  this  study.  Mean  age  of 
the  subject  group  was  26.7  (range:  21-38  years).  All  subjects  were  right-handed  and 
had  no  history  of  neurological  or  psychological  abnormality. 

b.  Task.  Stimuli,  subtending  a  visual  angle  under  1.5  degrees,  were  presented 
on  an  amber  CRT  monitor  placed  70  cm  from  subjects’  eyes.  Stimuli  had  a  duration 
of  315  msec  and  an  illumination  of  0.5  log  fL  against  a  background  of  -1.5  log  fL.  A 
visual  cue,  slanted  to  the  right  or  to  the  left,  indicated  to  subjects  to  prepare  to 
make  a  response  pressure  with  the  right  or  left  index  finger.  One  second  later,  the 
cue  was  followed  by  a  visual  numeric  stimulus  (number  1-9)  indicating  the  required 
pressure  of  the  response,  from  0.1  kg  to  0.9  kg.  Subjects  were  required  to  respond 
as  quickly  and  accurately  as  possible. 

Feedback  indicating  the  exact  response  pressure  produced  was  presented  as  a 
two-digit  number  one  second  after  the  peak  of  the  response  pressure.  The  feedback 
number  was  underlined  if  the  response  pressure  did  not  exceed  the  required 
pressure  by  more  than  a  threshold  based  on  previous  performance;  the  performance 
threshold  was  updated  on-line  after  each  trial  as  the  overage  error  from  the 
preceding  five  trials  for  each  hand  separately.  A  random  20%  of  the  trials  were  no¬ 
response  trials  in  which  the  slant  of  the  stimulus  was  opposite  to  that  of  the  cue 
and  no  response  was  to  be  made.  All  trials  had  a  duration  of  about  four  sec.  The 
inter-trial  interval  was  two  sec.  Between  eight  hundred  and  1000  trials  were 
performed  by  each  subject. 

c.  Recordings.  Twenty-six  channels  of  EEG  data  were  recorded  from  tin  disk 
scalp  electrodes.  Electrodes  were  placed  according  to  an  expanded  version  of  the 
standard  Ten-Twenty  Electrode  System  in  which  additional  coronal  rows  of 
electrodes  were  interposed  between  the  original  rows.  The  anterior  midline  parietal 
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electrode  was  used  as  the  reference.  EEGs,  as  well  as  vertical  and  horizontal  eye- 
movements  and  flexor  digitori  muscle  activity  from  both  arms,  were  digitized  at  128 
Hz  from  0.75  sec  before  the  cue  to  1.0  sec  after  feedback.  The  bandpass  had  a  6 
dB/octave  rolloff  below  0.1  Hz  and  a  24  dB/octave  rolloff  above  50  Hz.  The  rolloff 
below  0.1  Hz  was  gradual  enough  to  allow  sensitivity  to  ultrar-low  frequency  brain 
potential  components. 

The  scalp  potentials  were  converted  to  LDs  by  approximating  the  second  spatial 
derivative  of  the  potential  field  at  each  electrode.  All  single-trial  EEG  data  were 
screened  for  eye-movement,  muscle  potential  and  other  artifacts  by  two 
independent  editors.  Contaminated  data  (20%)  were  discarded. 

d.  Analyses.  Enhanced  averages  were  computed  by  eliminating  individual  trials 
that  lacked  discernible  event-related  signals  (18].  The  criterion  for  elimination  was 
based  on  a  "noise"  set,  composed  of  segments  of  ongoing  EEG  for  each  channel, 
randomly  timed  with  respect  to  any  stimulus  or  response  event.  The  distribution  of 
noise  segments  had  statistical  properties  similar  to  the  distribution  of  event-related 
segments.  Sets  of  event-related  and  "noise"  data  samples  were  submitted  to  a 
mathematical  pattern  classification  algorithm,  which  constructed  equations  to 
discriminate  between  the  two  sets  for  each  channel  in  a  brief  time  interval  (125  or 
250  msec).  Approximately  60%  of  the  data  remained  after  this  procedure.  An 
averaged  LD  waveform  over  an  entire  trial  is  shown  in  Figure  16.  ' 


RIGHT  MOVE,  CUE-LOCKED 
7  Hz  Filtered  Average 


Figure  16 

A  7-Ht  low-pas*  filtered,  Laplaclan  derivation  wive  form  (*CZ)  averted  across  seven  subjects  (or  s  full  four-sec  cued, 
bimanual  visuomotor  Usk  tried.  The  mldllae  antero-central  channel  (aCi)  is  shown. 
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Intervals  used  for  event-related  covariance  (ERC)  analysis  were  centered  on 
major  event-related  potential  peaks.  Bandpass  filter  characteristics  were  determined 
from  Wigner  time-frequency  distributions  of  the  enhanced  average  [19].  Covariances 
were  computed  between  each  of  the  120  pairwise  combinations  of  the  10  channels  in 
intervals  from  500  msec  before  cue  to  1000  sec  aitm  bao  .avuadbU. 

i.  Pr e-stimulus  ERCa:  Data  sets  were  separated  into  trials  in  which 
subsequent  performance  was  either  accurate  or  inaccurate.  Accurate  and  inaccurate 
performance  trials  were  those  in  which  the  error  (deviation  from  required  finger 
pressure)  was  less  than  or  greater  than,  respectively,  the  mean  error  over  the  entire 
recording  session.  Outlying  trials  on  the  distribution  of  performance  threshold  were 
eliminated  to  ensure  that  accurate  and  inaccurate  data  sets  did  not  differ  from  each 
other  due  to  learning  or  transitory  changes  in  arousal.  Accurate  and  inaccurate 
trials  were  evenly  distributed  throughout  the  recording  session. 

ERCs  were  computed  from  delta-bandpass-filtered  (0.01  to  3  Hz)  enhanced 
averages  for  a  brief,  375-msec-wide  interval  centered  on  the  Contingent  Negative 
Variation  (CNV)  (687  msec  poet-cue;  313  msec  prestimulus).  Covariance  functions 
were  computed  by  lagging  one  channel  of  the  pair  up  to  +/- 128  msec.  The  measure 
of  covariance  was  the  maximum  absolute  value  of  the  covariance  function.  The 
statistical  significance  of  each  covariance  was  assessed  by  reference  to  a  noise 
distribution  computed  from  the  same  data.  (Significant  ERCs  were  graphed  on  color 
three-dimensional  perspective  model  heads  and  published  in  [38]). 

ii.  Stimulus response -  and  feedback-registered  ERCs:  Analyses  of  the 
stimulus-,  response-  and  feedback-registered  events  were  performed  by  computing 
ERCs  from  187-msec-wide  intervals  centered  on  the  major  event-related  LD  peaks 
of  enhanced,  theta-band  filtered  (4  to  7  Hz)  averages,  including  the  Nl,  P300  and 
response  potentials.  Covariance  functions  were  computed  for  all  channel  pairs  by 
lagging  one  channel  of  the  pair  up  to  +/-  64  msec).  (Significant  ERCs  were  graphed 
on  color  three-dimensional  perspective  model  heads  and  will  be  published  soon 
[39,40].  Feedback  ERCs  were  computed  separately  for  accurate  and  inaccurate  trials 
as  described  above  for  the  prestimulus  ERCs. 

2.  Results  and  Discussion 

a.  Prestimulus  interval.  ERC  patterns  occurring  during  an  interval  spanning 
the  CNV  differentiated  subsequently  accurate  and  inaccurate  trials  (Fig.  17). 
Subsequently  accurate  performance  by  each  hand  was  characterized  by  significant 
ERCs  involving  electrodes  overlying  left-frontal  and  appropriately  lateralized  central 
and  parietal  cortices.  Inaccurate  performance  by  the  right  hand  was  preceded  by  a 
similar  pattern  but  with  fewer  covariances.  Inaccurate  performance  by  the  left  hand 
was  preceded  by  a  complex,  spatially  diffuse  ERC  pattern.  ERC  patterns  of 
subsequently  accurate  and  inaccurate  performance  for  both  hands  differed 
significantly  whereas  CNV  amplitudes  (Fig.  18),  per  se,  did  not. 
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Figure  17 

View  ol  the  moet  stgnlficint  (p< .06) ,  between -chmnel  ERG  pit  terns  from  ad  Intervsl  500  to  875  msec  After  the  cue  for 
subsequently  iccurite  And  iDACcurAte  left-hind  (A)  And  right-hind  (B)  performince  by  seven  right-hmded  subjects.  The 
thickness  of  a  line  Is  proportions)  to  Its  sign  15 cm ce  (from  .05  to  .005).  Line  color  tndicstes  the  sign  of  the  covsrisnce  (violet 
”  positive;  blue  as  negstlve),  Coviriinces  involving  left-trontil  And  Appropristely  lAterslited  centrsl  snd  pirietsl  electrode 
sites  sre  prominent  In  pitterns  for  subsequently  nccurste  performsnce  of  both  hsnds.  Msgnltude  snd  number  of  covsrlmces 
sre  greiier  preceding  subsequently  left-hind  Iniccuriie  performince;  fewer  snd  wesker  covirisnces  chirscterize 
subsequently  Insccurste  right- hind  performince. 


-38- 


LEFT  CUE.  ACCURATE 

F3.  AP2.  C3 

LEFT  CUE,  ^ACCURATE 

Fa.AP2.C3 

datta  Altar 

N  ■  262-274 

Mi  Attar 

N»  264-264 

(C) 

Figure  IS 

(D) 

Delta-filtered  Lap  laden  Derivation  waveforms  showing  the  CNV  in  the  cue-to-stiinuius  period  from  the  left-frontal  (F3), 
anterior  midline  central  (aCZ)  and  left  antero-parietal  (aPl)  sites  for  right-hand  accurate  (A)  and  inaccurate  (B) 
performance,  and  from  left-frontal,  midline  antero-central  and  right  antero-parietal  (aP2)  sites  for  left-hand  accurate  (C)  and 
inaccurate  performance  (D).  The  mean  amplitudes  of  the  late  CNV,  from  low-pass  filtered  (below  J  Hi)  averages  of  seven 
subjects,  did  not  differ  significantly  between  accuracy  conditions  for  either  hand  (3Sj. 

These  findings  axe  particularly  interesting  in  light  of  unresolved,  issues  concerning 
the  relationship  between  the  CNV  and  anticipatory  and  preparatory  processes 
[46,47].  One  important  issue  has  been  whether  this  slow  potential  solely  reflects 
motor  preparation  or  whether  a  cognitive  component  related  to  an  expected 
stimulus  is  also  involved.  In  addition,  among  attempts  to  relate  CNV  to  the  quality 
of  subsequent  performance,  even  the  most  recent  have  had  negative  or  mixed 
results  (e.g.,  [48])  Our  findings  indicate  that  the  CNV  is  a  multi-faceted  process 
with  components  that  function  synergistically  to  achieve  subsequently  accurate 
performance. 

We  speculate  that  the  prestimulus  ERC  patterns  reveal  the  existence  of  a 
preparatory  neural  set  that  involves  distinctive  cognitive  (frontal),  integrative- 
motor  and  lateralized  somesthetic-motor  components.  The  involvement  of  the  left- 
frontal  site  is  consistent  with  clinical  findings  that  preparatory  sets  are  synthesized 
and  integrated  in  prefrontal  cortical  areas  (reviewed  in  [49])  and  with  experimental 
and  clinical  evidence  indicating  involvement  of  the  left  dorsolateral  prefrontal  cortex 
in  delayed  response  tasks.  A  midllnf-  antero-central  integrative  motor  component  is 
consistent  with  known  involvement  of  premotor  and  supplementary  motor  areas  in 
initiating  motor  responses  [50,51],  The  finding  of  an  appropriately  lateralized  central 
and  parietal  component  is  consistent  with  evidence  from  primates  and  humans  for 
neuronal  Firing  in  motor  and  somatosensory  cortices  prior  to  motor  responses  (e.g., 
[52,53]). 

The  fact  that  stronger  ERC  patterns  were  found  for  subsequently  inaccurate  left- 
hand  than  for  inaccurate  right-hand  performance,  particularly  for  sites  overlying 
motor  areas,  may  indicate  that  a  'stronger*  preparatory  set  is  required  for  left-hand 
performance  by  strongly  right-handed  subjects.  In  a  study  of  rhesus  monkeys, 
Johnson  [53]  reported  that  slow  potential  activity  for  the  initiation  of  voluntary 
movement  begins  earlier,  and  is  present  over  medial  prefrontal  cortex  bilaterally,  for 
movement  of  a  non-preferred  hand  relative  to  a  preferred  hand.  The  generally 
diffuse  ERC  patterns  for  subsequently  inaccurate  left-hand  performance  compared 
with  the  right  may  also  suggest  that  inaccurate  left-hand  performance  results  from 
erroneous  or  confounded  preparatory  sets. 

b.  Stimulus  interval.  Response  and  no-response  conditions  for  right-  and  left- 
slanted  stimuli  were  examined  in  early  and  late  poststimulus  intervals.  For  the 
interval  on  the  N1  peak,  ERG  patterns  (Fig.  19)  were  differentiated  according  to 
the  direction  of  stimulus  slant,  but  not  to  response  condition.  The  right  parietal  site 
was  involved  in  patterns  for  both  right-  and  left-slanted  stimuli,  perhaps  reflecting 
the  importance  of  that  region  in  attentional  tasks  [54].  An  additional  parietal 
component,  ipsilateral  to  the  stimulus  slant,  appeared  to  represent  an  obliquely 
oriented  contralateral  occipital  generator  that  projected  to  the  opposite  hemisphere. 

ERCs  in  the  interval  centered  at  312  msec  on  the  P300  peak  differed 
substantially  between  response  conditions  (Fig.  20),  but  no  longer  between  stimulus 
conditions.  The  midline  antero-central  site  (aCZ)  was  prominently  involved  in  ERC 
patterns  for  the  no-response  condition  in  this  interval,  possibly  reflecting 
involvement  of  the  underlying  supplementary  motor  and  premotor  cortices  in  motor 
inhibition. 
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Fleur*  IS 

View  of  the  mo*t  rignlficunt  (top  standard  deviation)  ERC  pattern*  (colored  line*),  looking  down  at  the  top  of  the  head, 
from  the  stimulus-locked  waveform  of  the  response  condition,  superimposed  on  colored  maps  of  Nl  wave  amplitude.  The 
event-related  covariance*  were  measured  during  a  187-msec-wide  interval,  centered  62  msec  after  left  and  right  stimulus 
onset,  from  theta-band  titered,  7-subject  averages.  The  thickness  of  a  covariance  line  is  proportional  to  the  negative  log  of 
its  significance  (from  .05  to  .00005).  The  color  scale  at  the  left,  representing  wave  amplitude,  covers  the  range  from  the 
minimal  to  maximal  value*  of  the  two  maps.  For  both  left-  and  right-handed  conditions,  there  is  a  maximum  of  wave 
amplitude  occipitally  and  a  minimum  at  the  antero-ccntral  site.  Common  covariance*  for  both  left-  and  right-band 
conditions  exist  between  midline  posterior  parietal  (Pi)  and  antero-parietal  (aPi),  and  between  midline  antero-parietsi  and 
antero-central  (aCi)  sites.  Additional  covariance*  tor  the  right-hand  condition,  involving  the  right  parietal  (P4)  site,  reflect 
the  lateraliiaiion  of  the  Nl  peak  at  parietal  site*. 
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View  o(  the  meat  significant  (top  standard  deviation),  P300  ERC  pattern*  (colored  line*),  looking  down  at  the  top  of  the 
head,  superimposed  on  colored  map*  of  Integrated  P300  amplitude.  Measurement*  are  from  an  interval  218  to  406  msec  after 
right-cue  stimulus  onset,  tor  response  (TRESS*  tor  pressure)  and  uo- response  ('NO  PRESS*  tor  no  pressure)  conditions,  on 
the  P300  peak  of  theta-band  filtered,  7-subject  averages.  The  midUne  parietal  site  (Ft)  is  central  to  the  pattern  for  the 
response  condition,  and  the  midline  antero-central  site  (aCt)  is  central  for  the  infrequent  no-response  condition.  Pi  lags 
most  other  sites  in  the  response  condition,  while  aCi  lags  in  no-response. 


c.  Response  interval.  Differences  between  right-  and  ieft-hand  response 
conditions  were  examined  in  intervals  before  and  after  response  onset.  At  the  peak 
of  the  finger  pressure  (62  msec  after  response  onset;  Fig.  2l),  ERC  patterns  were 
appropriately  contralateral  to  the  hand  used  (Fig.  22).  The  midline  antero-central 
site  was  prominent  in  all  patterns.  The  sign  of  the  ERCs  of  this  site  with  posterior, 
left-central  (C3)  and  antero-parietd  (aPl)  sites  for  right  movement,  and  right- 
central  (C4)  and  antero-parietal  (aP2)  sites  for  left  movement,  was  opposite  to  that 
with  anterior  sites,  reflecting  the  transition  from  an  anterior  current  sink  to  a 
posterior  current  source.  The  lack  of  significant  ERCs  betwetn  lateral  central  and 
antero-parietal  sites  suggests  the  existence  of  separate  generators  on  either  side  of 
the  central  sulcus  covarying  with  a  third  generator  under  the  midline  antero-central 
site,  possibly  in  supplementary  motor  cortex. 


RIGHT  MOVE-LOCKED  LEFT  MOVE-LOCKED 
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Response-locked,  Laplacian  Derivation  waveforms  (or  right-hand  (A)  and  left-hand  (B)  responses  from  midline  antero-central 
(aCZ),  left-  and  right-central  (C3,  04)  sites  averaged  across  seven  subjects.  Emerging  'equivalent  current"  is  up. 
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Figure  » 

View  of  the  meet  significant,  ERC  patterns  from  the  wave  at  the  peak  of  the  finger  response.  The  motor-related  wave  was 
measured  during  a  187-msec  Interval  centered  on  the  peak  of  the  left-hand  and  right-hand  Index  finger  pressures  from 
thetsrband  filtered,  seven-subject  averages,  The  thickness  of  a  line  Is  proportional  to  its  significance  (from  .06  to  .00005), 
Line  color  indicates  the  time  delay  (lag  time  of  maximum  covariance),  and  the  arrow  points  from  the  leading  to  the  lagging 
channel.  Colored  arrows  or  boxes  indicate  positive  covariances,  while  tan  arrows  or  boxes  Indicate  negative  covariances. 
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d.  Feedback  interval.  Distinct  ERC  patterns  associated  with  visual  feedback 
about  accuracy  of  response  performance  were  most  clearly  defined  in  the  187-msec¬ 
wide  interval,  centered  375  msec  post-feedback  (Fig.  23).  Prior  to  this  interval  (in 
intervals  centered  125  msec  and  250  msec  poet-feedback,  for  example),  ERGs  for 
both  hands,  irrespective  of  accuracy,  involved  almost  exclusively  the  posterior, 
central,  and  antero-central  midline  channels.  Beginning  with  the  interval  centered 
at  375  msec,  a  pattern  involving  ERCs  from  midline  occipital  and  parietal  to 
midline  antero-central  and  right-frontal  sites  was  common  to  both  accurate  and 
inaccurate  right-  and  left-hand  responses.  When  responses  were  inaccurate,  the 
feedback  pattern  additionally  included  the  left  antero-central  site  for  the  right  hand 
and  the  left-frontal  site  for  the  left  hand.  Therefore,  while  feedback  to  accurate 
performance  involved  xnidline  sites  and  sites  overlying  the  right  frontal  cortex, 
feedback  to  inaccurate  performance  was  characterized  by  a  more  symmetric  ERC 
pattern  involving  sites  overlying  frontal,  central  and  parietal  areas  bilaterally.  These 
patterns  were  not  apparent  in  topographic  maps  of  the  average  ERPs. 
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Figure  21 


View  of  the  most  significant  (top  two  standard  deviations),  ERC  patterns  (colored  linesi,  looking  down  at  the  top  of  the 
head,  tor  feedback  to  accurate  and  Inaccurate  performance  by  the  left  band  (A  and  Bjand  right  hand  (C  and  D).  ERC 
pattern  are  superimposed  on  color  mapi  of  wave  amplitude,  The  thickness  of  a  covariance  line  if  proportional  to  tbe 
negative  log  of  It*  significance  (from  ,05  to  .00005).  The  color  scale  at  tbe  left  of  each  picture,  representing  wave  amplitude, 
covers  the  range  from  the  minimal  to  maximal  values  of  the  two  maps.  Both  tbe  covariances  and  tbe  amplitude  maps  are 
derived  from  a  187-msec-wlde  interval,  centered  at  375  msec  after  feedback  ouset,  on  tbeta-baud-filtered  7-subject  averaged 
waveforms.  Patterns  were  similar  between  hands  and  conditions  in  an  early  Interval  centered  at  125  msec,  and  began  to 
differ  in  the  interval  centered  at  287  msec  which  spanned  part  of  tbe  early  P300.  For  tbe  Interval  centered  at  375  msec 
shown  in  the  figure,  the  patterns  for  feedback  to  accurate  performance  involve  tbe  right  frontal  and  antero-central  sites,  but 
not  the  left.  Tbe  patterns  tor  feedback  to  inaccurate  performance  additionally  involve  the  left  frontal  and  antero-central 
sites,  and  the  right  parietal  site.  These  sites  may  reflect  additional  attention  necessary  to  improve  performance  on 
subsequent  trials, 


Previous  evoked  potential  studies  of  feedback  about  performance  accuracy  have 
suggested  that  evaluation  of  feedback  information  may  be  a  serial  process  [55,56],  It 
has  been  suggested  that  an  early  P300  is  associated  with  the  identification  or 
classification  of  feedback  and  that  a  later  positive  wave  is  associated  with  the 
subsequent  utilization  of  that  feedback  information.  Our  results  provide  further 
information  about  the  dynamic  temporal  and  spatial  processes  related  to  feedback 
about  performance  accuracy.  Involvement  of  the  midline  antero-central  site 
(overlying  the  supplementary  motor  area)  following  feedback  to  both  accurate  and 
inaccurate  performance  may  reflect  "motor  recalibration"  consequent  to  feedback 
information.  The  involvement  of  frontal  sites  may  reflect  feedback-specific 
"updating"  and  behavioral  verification.  These  interpretations  are  consistent  with 
experimental  and  clinical  evidence  suggesting  that  frontal  and  supplementary  motor 
cortices  are  involved  in  planning  and  sequencing  voluntary  behavior  [49]. 
Involvement  of  the  left-frontal  site  specifically  for  feedback  about  inaccurate 
performance  is  particularly  interesting  because  its  involvement  is  also  a 
distinguishing  feature  of  the  preparation  period  preceding  accurate  performance. 

In  conclusion,  the  measurement  of  event-related,  between-channel  covariances 
provides  insights  into  the  spatiotemporal  characteristics  of  preparation,  stimulus 
processing,  response  execution  and  processing  of  feedback  information  in  a  difficult 
visuomotor  task.  Taken  together,  the  ERC  patterns  appear  to  reflect  the  co¬ 
ordination  of  specific  neural  regions.  Although  the  origin  of  these  patterns  is  not  yet 
clearly  understood,  their  consistency  with  previous  neuroanatomical  models  suggests 
that  they  primarily  reflect  cortical  activity.  The  focal,  spatially  separated  patterns 
tend  to  rule  out  an  explanation  based  solely  in  terms  of  volume-conducted  activity 
from  one  or  two  distant  subcortical  or  cortical  generators.  In  view  of  the  striking 
findings,  determining  the  actual  source  of  these  patterns  is  a  compelling  issue. 


B.  Fatigue  Experiment 
1.  Overview 

^paired  behavior  consequent  to  prolonged  mental  work  is  commonly  attributed 
to  the  effects  of  fatigue  on  higher  cognitive  functions  rather  than  to  changes  in  rote 
perceptuomotor  or  motor  functions.  A  deeper  understanding  of  these  effects  awaits 
better  knowledge  of  the  underlying  neurophysiological  mechanisms.  Towards  this 
end,  event-related,  spatiotemporal  neuroelectric  patterns  of  five  U.S.  Air  Force  test 
pilots  performing  a  high-load  visuomotor  monitoring  task  while  they  were  alert, 
becoming  fatigued,  and  fully  fatigued.  These  results  suggest  that  the  effects  of 
mental  fatigue  are  not  only  evident  in  spatially-  and  temporally-specific 
neuroelectric  patterns  prior  to  overt  signals  of  behavioral  deterioration,  but  that 
cortical  networks  underlying  certain  higher  cognitive  functions  are  selectively 
altered  during  the  incipient  stages  of  fatigue  while  those  involved  in  visual  stimulus 
processing  are  relatively  unaffected.  With  further  research,  it  may  be  possible  to 
design  on-line  devices  for  predicting  'transient  cognitive  lapses'  and  performance 
decrements  resulting  from  operational  fatigue. 


2.  Introduction 

It  is  well  known  that  higher  cognitive  functions  are  more  affected  by  fatigue  due 
to  prolonged,  difficult  mental  work,  than  are  rote  perceptuomotor  or  motor 
functions.  Although  psychological  theories  have  attempted  to  explain  these  changes, 
numerous  reports  of  tragic  accidents  caused  by  fatigued  pilots  and  operators  of 
other  complex  equipment  clearly  demonstrate  the  need  for  a  better  understanding 
of  these  phenomena.  Here  we  report  a  study  of  event-related,  spatiotemporal 
neuroelectric  patterns  associated  with  visual  stimulus  processing  and  response 
inhibition:  five  U.S.  Air  Force  test  pilots  performed  a  high-load  visuomotor  memory 
task  while  alert,  becoming  fatigued,  and  fully  fatigued. 

The  study  emerged  from  the  collaboration  of  researchers  from  several  institutions: 
(1)  The  USAF  School  of  Aerospace  Medicine,  San  Antonio  (James  Miller);  (2| 
Systems  Technology  Inc.,  Hawthorne,  California  (Henry  Jex  and  James  Smith);  (3) 
Washington  University,  St.  Louis  (John  Stem);  and  (4)  The  EEG  Systems 
Laboratory.  A  paradigm  was  developed  that  included  severed  cognitive  and 
perceptuomotor  tasks  requiring  varying  degrees  of  attention,  working  memory,  and 
skill.  Thirty-three  or  51  scalp  EEG,  two  eye-movement,  and  several  other 
physiological  channels  of  data  were  recorded  from  each  of  the  five  test  pilots  as  they 
performed  the  task  battery. 

3.  Methods 

a.  Requirements.  The  present  experiment  took  into  account  certain  criteria 
that  must  be  met  in  order  to  isolate  the  effects  of  operational  fatigue  on 
performance  to  ensure  that  the  results  would  have  a  modicum  of  construct  and  face 
validity  for  real-life  situations,  such  as  piloting  an  aircraft.  These  criteria  included: 
(l)  tasks  that  are  taxing  in  an  alert  state,  yet  manageable  hi  a  fatigued  state;  (2) 
tasks  meeting  constraints  imposed  by  neurophysiological  recordings  and  analysis, 
including  minimal  eye-  and  head-movements,  and  isolation  of  stimulus,  response, 
and  cognitive  variables  [3,4,57,58];  (3)  experimental  paradigms  controlled  for  initial 
task  learning  and  automatization  of  performance;  and  (4)  recording  sessions  that 
extend  from  well-practiced,  alert  performance  to  fatigued  and  decremented 
performance  due  to  extended  time-on-task. 

b.  Experimental  Protocol.  The  experiment  was  conducted  over  the  course  of 
four  days.  Pilots  learned  and  performed  a  battery  of  tasks  for  about  6  hours  on  the 
first  day  until  their  performance  stablized  (Session  1).  They  returned  to  the 
laboratory  the  following  morning,  were  prepared  for  recording,  and  began 
performing  the  tasks  at  about  1:30  PM  (Session  2).  After  a  dinner  break  at  about 
8:00  PM,  they  resumed  task  performance  for  an  additional  0-8  hours  (Session  3) 
until  they  could  no  longer  continue,  or  usable  EEG  data  could  no  longer  be 
recorded  because  of  excessive  eye  movements  or  head  nodding  and  drooping. 

Recordings  were  made  in  an  acoustically  dampened,  air-conditioned  chamber. 
Room  temperature,  lighting,  and  seating  position  were  adjusted  as  necessary. 

c.  Task  Battery.  The  total  task  battery  consisted  of  five  tasks:  (1)  Visuomotor 
Monitoring  Task,  null  memory  condition;  (2)  Visuomotor  Memory  Task,  memory 


lead  condition  (called  VMMT  in  this  report);  (3)  Auditory  Monitoring  Memory 
(AUM)  Task;  (4)  Sub-critical  Tracking;  and  (5)  Critical  Tracking  [59,60],  The  four 
tasks  alternated  within  experimental  units  each  lasting  about  two  hours.  The 
analysis  reported  here  focused  exclusively  on  the  difficult  VMMT.  Results  for  the 
other  tasks  will  be  described  elsewhere. 

The  VMMT  constituted  about  40%  of  each  two-hour  unit.  It  was  designed  to 
allow  for  precise  control  of  stimulus  parameters,  motor  activity  including  eye 
movements,  and  cognitive  processes.  This  was  done  by  combining  a  numeric 
judgment  task  used  previously  [6,38]  with  a  memory  element  based  on  the  "delayed 
digit  cancelling  task."  It  required  that  subjects  hold  a  stimulus  number  in  memory, 
despite  numeric  distractors,  to  produce  a  precise  finger  pressure  corresponding  to 
that  number  two  trials  later  (Fig.  24).  Each  trial  began  with  a  warning:  the 
disappearance  of  the  letter  X  from  the  video  screen.  This  was  followed  0.75  second 
later  by  the  appearance  of  a  single-digit  stimulus  number  to  be  remembered. 
Subjects  were  then  required  to  respond  to  the  stimulus  number  presented  two  trials 
back.  This  response  consisted  of  a  flexion  of  the  right  index  finger  on  an  isometric 
transducer  with  a  pressure  of  0.1  to  0.9  kg.  One  second  after  the  peak  of  the 
response,  a  two-digit  feedback  number  was  presented  indicating  the  applied  pressure 
with  a  precision  of  0.01  kg.  For  example,  if  the  stimulus  numbers  in  five  successive 
trials  were  8, 6, 1,9, 4,  the  correct  finger-pressure  responses  would  be  0.8  kg  to  the  1, 
0.6  kg  to  the  9,  and  0.1  kg  to  the  4. 


VISUOMOTOR  MEMORY  TASK  (VMMT) 
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remembered,  followed  by  the  subject's  finger-pressure  response  to  the  stimulus  number  presented  two  trills  back,  followed  1 
second  after  the  peak  of  the  response  by  presentation  of  a  2-digit  feedback  number  indicating  the  accuracy  of  the  response. 
If  the  subject's  response  was  highly  accurate,  the  feedback  number  was  underlined.  To  further  increase  the  task  difficulty, 
subjects  were  required  to  withhold  their  response  when  the  current  stimulus  number  was  identical  to  the  stimulus  two  trials 
back.  (No  response  was  required  on  the  first  two  trials  of  each  block  of  50  trials  —  signified  by  'OK'). 
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To  further  increase  the  task  difficulty,  subjects  were  required  to  withhold  their 
response  whenever  the  current  number  was  identical  to  the  number  two  trials  back. 
For  example,  presented  with  9,4,9  as  the  stimulus  numbers  in  three  successive 
trials,  the  subject  was  to  inhibit  his  finger  response  to  the  second  nine.  These  no¬ 
response  trials  comprised  a  random  20%  of  the  trials. 

Stimuli  were  presented  for  325  msec  on  a  Videographics-H  amber  CRT  monitor  70 
cm  from  the  subject.  Stimuli  subtended  a  visual  angle  under  1.5  degrees,  with  an 
illumination  of  0.5  log  fL  against  a  background  of  -1.5  log  fL.  The  stimulus  and  ' 
feedback  numbers  were  given  in  'response  units",  10  times  the  target  and  actual 
pressures  (in  kg),  respectively.  The  feedback  number  was  underlined  if  the  response 
was  a  "win."  A  "win*  response  was  based  on  an  adaptive  error  tolerance,  computed 
as  the  geometric  average  of  the  error  (distance  from  the  required  response  pressure) 
on  the  previous  five  trials  [6].  This  adaptive  error  tolerance  served  to  equalize 
subjective  task  difficulty  across  the  session,  to  indicate  the  current  performance 
trend,  and  to  increase  the  difficulty  of  earning  "bonus  points"  during  episodes  of 
improving  performance.  At  the  end  of  each  block  of  50  trials,  subjects  were  given  a 
block  report  indicating  their  performance  accuracy,  number  of  time-outs  and  false 
responses,  and  bonus  points  earned. 

< 

There  was  0.75  sec  from  the  warning  (X  off)  to  stimulus  onset,  an  average  of  0.54 
sec  to  response  onset,  and  average  of  0.07  sec  to  response  peak,  1.00  sec  to  feedback 
onset,  then  1.25  sec  to  the  end  of  trial  (X  on),  and  then  1.80  sec  inter-trial  interval. 
Thus,  the  time  from  one  warning  to  the  next  was  5.4  sec,  and  from  one  stimulus  to 
its  response  two  trials  later  was  11.4  sec.  This  pace  was  fast  enough  to  minimize 
rehearsal,  while  still  allowing  the  well-practiced  subjects  to  maintain  the  numbers  in 
working  memory  [61]. 

During  pilot  studies,  subjects  commonly  reported  that,  after  practicing  the  task, 
they  developed  an  accurate  "sense"  for  the  2-back  number.  Several  of  the  subjects 
also  developed  faster  reaction  times  in  the  memory  task  than  in  the  comparable 
non-memory  task,  yet  exhibited  no  deficiency  in  detecting  stimulus  matches  on  the 
no-response  trials.  This  suggested  a  priming  of  the  motor  system  (since  the 
required  two-back  number  was  known),  with  the  ability  to  abort  this  preparatory 
facilitation  upon  recognition  of  a  two-back  match. 

d.  Physiological  and  Anatomical  Recording  Methods.  Fifty-one  EEC  channels 
were  recorded  for  three  of  the  five  subjects  with  a  custom  nylon  mesh  cap. 
Recordings  of  the  other  two  subjects  were  made  with  a  reduced  33-channel 
montage.  All  signals  were  amplified  by  a  64-channel  Bioelectric  Systems  Model 
AS-64P  with  .016  to  50  Hz  passband,  monitored  on  three  8-channel  and  two  16- 
channel  polygraphs,  and  digitized  to  11  bits  at  128  Hz.  Vertical  and  horizontal  eye 
movements  (VEO,  HEO),  flexor  digitorus  muscle  (EMG),  cardiac  rate  (EKG), 
respiration,  and  left  mastoid  were  recorded  from  all  subjects.  For  all  five  subjects, 
recordings  were  made  with  the  following  set  of  33  scalp  channels  referenced  to  the 
right  mastoid:  left  and  right  antero-frontal  (aFl,  aF2),  left  and  right  inferior  frontal 
(F7,  F8),  left  and  right  superior  frontal  (F3,  F4),  midline  frontal  (Fz),  left  and  right 
antero-central  (aC3,  aC4),  midline  antero-central  (aCz),  left  and  right  inferior 
temporal  (T5,  T6),  left  and  right  superior  temporal  (T3,  T4),  left  and  right  central 
(C3,  C4),  midline  central  (Cz),  left  and  right  inferior  antero-parietal  (aP3,  aP4),  left 


and  right  superior  antero-parietal  (aPl,  aP2),  left  and  right  mid-parietal  (P3,  P4), 
midline  mid-parietal  (Pz),  left  and  right  postero-parietal  (pPl,  pP2),  midline 
occipital  (OZ). 

Before  and  after  a  recording  session,  the  position  of  each  electrode  was  measured 
using  a  3-D  digitizer.  A  general  ellipsoid  surface  was  fitted  to  the  set  of  digitized 
positions,  thus  defining  the  outside  shape  of  the  head.  For  the  three  subjects  with 
51-channel  recordings,  full  sets  of  3-axis  MR  imago  brain  scans  were  made  at  1-cm 
intervals  over  the  whole  volume  of  the  head.  After  scaling,  translation  and 
registration  with  the  EEG  electrode  positions,  the  MRIs  allowed  correlation  of 
electrode  positions  and  major  cortical  landmarks  [8,62]. 

e.  Reduction  of  Volume  Conduction  Blurring  of  EEGs.  An  optimal  least- 
squares  estimate  of  the  Laplacian  operator  was  applied  to  the  EEGs  to  eliminate 
dependence  on  the  recording  reference  electrode  and  to  compensate  for  the  blurring 
of  brain  potentials  due  to  their  transmission  through  the  skull  and  scalp  [8,10]. 
Because  estimating  the  Laplacian  at  peripheral  electrodes  is  problematic,  they  were 
eliminated  from  the  analysis.  This  left  18  remaining  non-peripheral  electrodes 
common  to  all  five  subjects. 

f.  Formation  of  Data  Sets.  Polygraph  records  were  edited  off-line  by  two 
independent  raters  to  eliminate  trials  with  evidence  of  eye  movement,  muscle  or 
instrumental  artifacts.  Trials  in  which  response  pressures  were  not  unimodal  or  in 
which  reaction  time  exceeded  1500  msec  were  discarded.  The  first  two  trials  of  each 
block,  and  no-response  trials  in  which  the  subject  responded,  were  also  eliminated. 

VMMT  trials  were  divided  into  "alertness'’  (that  is,  "Alert,"  "Incipient  Fatigue* 
and  "Full  Fatigue")  conditions.  Behavioral  variables  (including  adaptive  error 
tolerance,  response  error,  reaction  time,  number  of  time  outs  on  response  trials,  and 
number  of  responses  on  trials)  were  graphed  across  trials  for  each  session.  The 
"Alert"  data  set  consisted  of  trials  from  Session  2  that  had  optimal  performance  as 
evidenced  by:  (l)  low  adaptive  error  tolerance  on  response  trials;  (2)  zero  time-outs 
on  response  trials;  and  (3)  zero  false  responses  on  response-inhibition  trials.  Trials 
from  early  portions  of  Session  3  with  little  or  no  performance  decrement  formed  the 
"Incipient  Fatigue*  data  set.  Trials  occurring  later  in  Session  3  formed  the  "Full 
Fatigue"  data  set. 

We  decided  to  select  only  the  most  accurate  trials  from  the  Alert  data  set,  and 
trials  with  comparable  accuracy  from  the  Incipient  Fatigue  and  Full  Fatigue  data 
sets,  because  we  previously  found  clearcut  neuroeiectric  patterns  specific  to  accurate 
performance  [38,40].  Selection  of  the  most  accurate  trials  was  done  for  each  subject 
individually,  with  the  ADISORT  program.  Using  this  interactive  program,  the 
three  trial  sets  were  divided  into  subsets  with  relatively  accurate  or  inaccurate 
responses,  in  which  the  error  was  less  or  greater  than,  respectively,  the  mean  error 
of  the  Alert  condition  trials.  Then  the  least  accurate  trials  of  the  accurate  Incipient 
Fatigue  and  Full  Fatigue  subsets  (roughly  2-4%  of  the  trials  of  each)  were 
discarded,  making  performance  equivalent  (p>0.2)  between  each  of  the  Fatigue 
accurate  subsets  and  the  Alert  accurate  subset. 

The  resulting  performance  equivalence  allowed  for  rigorous  testing  of  the 
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neuroelectric  signs  of  incipient  fatigue  while  controlling  for  effects  due  to  small 
variations  in  performance.  Trials  for  Alert,  Incipient  Fatigue,  and  Full  Fatigue 
conditions  were  performed  from  about  1:30  PM  to  8:00  PM,  from  about  8:30  PM  to 
10:30  PM,  and  from  about  11:30  PM  to  1:30  AM,  respectively.  According  to  a  7- 
point  U.S.  Air  Force  School  of  Aerospace  Medicine  Subjective  Fatigue  Rating  Scale 
(with‘1  corresponding  to  maximal  alertness  and  7  to  complete  exhaustion),  the 
average  ratings  of  the  five  pilots  for  the  three  conditions  were  3.44  +/-  1.85,  5.25 
+/-  1.42  and  6.23  +/-  0.65. 

g.  Averaged  LDs.  Grand-average  (over  the  five  pilots)  LDs  from  event-related 
potentials  were  computed  time-locked  to  the  stimulus.  Because  each  pair  of  data 
sets  was  balanced  behaviorally,  we  could  subtract  the  LDs  of  the  Incipient  Fatigue 
set  from  those  of  the  Alert  set,  and  of  the  Full  Fatigue  set  from  those  of  the 
Incipient  Fatigue,  to  determine  the  task-related  neuroelectric  effects  associated  with 
fatigue,  unconfounded  by  trends  of  declining  performance.  Use  of  the  subtraction 
technique  required  that  several  conditions  be  met,  including  the  above-mentioned 
experimental  controls,  as  well  as  the  absence  of  large  shifts  in  the  latency  of  peaks 
between  the  two  conditions.  By  subtracting  the  LDs  of  one  set  from  the  other, 
differences  between  them  were  highlighted  and  common  elements  were  cancelled 
out. 


h.  Event- Related  Covariances  (ERCs).  Spatiotemporal  neurophysiological 
patterns  were  quantified  by  measuring  the  event-related  covariance  (similarity  of 
amplitude-weighted  waveshape  and  between-channel  timing)  between  averaged 
waveforms  from  all  pairwise  combinations  of  electrodes.  This  analysis  was 
performed  with  the  methods  of  Section  HIB  (see  also  5-8,38-41,62,63], 

Changes  from  Alert  to  Incipient  and  from  Incipient  to  Full  Fatigue  conditions 
were  assessed  by  computing  ERCs  across  brief  intervals  of  the  appropriate  five- 
subject  grand-averaged,  bandpass-filtered,  subtraction  LDs.  ERCs  were  measured 
between  all  153  pairwise  combinations.  The  first  ERC  interval,  applied  to  the 
delta-bandpass-filtered  (0.01  to  3  Hz)  LD  of  the  finger-pressure  response  trials,  was 
500  msec  wide  and  was  centered  on  the  CNV  (312  msec  before  presentation  of  the 
numeric  stimulus).  The  next  two  ERC  intervals  were  187  msec  wide  and  were 
centered  on  the  thetarbandpass-filtered  (4  to  7  Hz)  N125  and  P380  peaks  (of  the 
response  and  response-inhibition  averages,  respectively)  elicited  by  the  numeric 
stimulus. 

ERC  features  were  the  maximum  absolute  value  of  the  timeseries  crosscovariance 
function  formed  when  the  covariance  function  was  computed  to  16  lags  (+/-128 
msec)  for  the  delta-band-filtered  prestimulus  interval  and  8  lags  (+/-64  msec)  for 
the  theta-band-filtcred  poatstimulus  intervals.  The  significance  of  the  ERC 
magnitude  was  determined  by  reference  to  the  standard  deviation  of  an  ERC 
'noise'  distribution  obtained  from  the  set  of  alert  trials.  Multiple  comparisons  were 
accounted  for  using  a  Duncan  procedure.  All  significant  ERCs  in  each  interval  were 
graphed  on  3-D  perspective  models  of  the  head,  or  on  sagittal  or  horizontal  views  of 
the  MR  scans.  The  lag  time  of  the  maximum  covariance  was  not  examined  for 
ERCs  because  of  the  difficulty  of  interpreting  the  delay  between  two  subtraction 
LDs. 


4.  Results 


a.  Behavioral.  In  the  original  data  sets,  performance  declined  significantly  over 
the  three  conditions  (F-12.5,  df=2,12,  pC.OOl)  (Fig.  25).  Scheffe  post-hoc  tests 
revealed  that  performance  did  not  decline  significantly  from  Alert  (1.17  unit 
.  departure  from  target  number)  to  Incipient  (1.51)  conditions,  but  did  decline 
significantly  from  Incipient  to  Full  Fatigue  (2.08)  conditions  (F=12.28,  df=2,12, 
p<0.001).  The  number  of  false  responses  during  no-response  trials,  as  well  as  the 
number  of  time-outs  on  response  trials,  also  tended  to  increase  across  alertness 
conditions.  The  increase  in  the  number  of  false  responses  was  significant  (F=17.5, 
df=2,12,  p<0.001),  and  Scheffe  post-hoc  tests  revealed  that  the  number  of  false 
responses  increased  between  Alert  and  Full  Fatigue  conditions  (F=15.96,  df=2,12, 
p<0.001),  and  between  Incipient  Fatigue  and  Full  Fatigue  conditions  (F=9.42, 
df=2,12,  p<0.003).  Although  the  number  of  time-outs  tended  to  increase  with 
increasing  fatigue  for  all  subjects,  the  effect  was  not  significant. 


.  F  A  T  I  e  U  E  EXFE«)flENT  MMif!" 


Figure  IS 

Performance  degradation  shown  u  participants  fatigued.  Eric t ton  time  end  error  both  increased  with  time-on-task.  The 
degradation  accelerated  substantially  in  this  participant  after  about  10  hours  of  performance.  Alert  trial?  were  taken  from 
the  first  seven  hourt  in  Session  2  when  performance  had  not  yet  degraded;  Incipient  Fatigue  trials  from  the  following  two 
hours  when  performance  was  jusi  beginning  to  degTtde  {end  of  Session  2  and  beginning  of  Session  3);  and  Full  Fatiuge  trials 
from  the  last  several  hours  of  Session  3  when  performance  was  most  degraded. 


b.  Neurophysiological.  The  root-mean-square  amplitude  of  the  CNV,  over  the 
500-msec  prestimulus  interval  (Fig.  26 A),  did  not  differ  across  alertness  conditions. 
There  was  a  marked  decrease  at  one  channel  (midline  central),  however,  from  Alert 
(5.2  mV/  cm2  )  to  Incipient  Fatigue  (1.7  mV/  cm2  ).  Yet  at  another  channel  (left 
superior  antero-parietal)  there  was  a  slight  increase  from  Alert  (2.1  fiV  /  cm  )  to 
Incipient  Fatigue  (2.8  mV/  cm2 ). 


Fleur*  MA 

Grand-average  (over  five  subjects),  LD  waveforms  of  the  mid  11  tie  central  (Cz)  electrode  during  the  vtsuomotor-memory  task 
for  Alert  (S&4  trials)  (dashed  Use),  Incipient  Fatigue  (617  trials)  (dotted  line),  and  for  Alert- minus- In  clpient-Fatlgue  (solid 
line).  The  mu  amplitude  from  562  to  62  msec  before  the  stimulus  did  not  significantly  differ  between  Alert  and  Incipient 
Fatigue  conditions. 


A  number  of  significant  Alert-minus-Incipient-Fatigue  ERCs  were  found  during 
the  same  prestimulus  interval.  Four  of  the  five  significant  ERCs  involved  the 
midline  central  electrode  (Fig.  26B).  ERCs  between  midline  central  and  left  mid' 
parietal,  and  left  superior  antero-parietal  electrodes  were  strongest,  with  additional 
significant  covariances  involving  right  superior  antero-  and  poetero-parietal 
electrodes.  The  values  of  the  five  significant  ERCs  were:  -44  (midline  central  to  left 
mid-parietal),  -40  (midline  central  to  left  superior  antero-parietal),  -31  (midline 
central  to  right  superior  antero-parietal),  -30  (midline  central  to  right  postero- 
parietal),  and  27  (left  mid-parietal  to  left  superior  antero-parietal).  The  ERC 
‘'noise"  distribution  had  a  mean  of  0  and  a  standard  deviation  of  6.3.  There  were 
no  significant  ERCs  for  the  Incipient-minus-Full-Fatigue  subtraction. 
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Figure  MB 

Event-related  covariance  (ERC)  patterns  characterising  the  prestlmulus  difference  between  Alert  and  Incipient  Fatigue 
conditions  from  front  (left)  and  top-of-head  (right)  perspectives.  ERCs  were  computed  from  Alert-minus-lncipicnt-Fatigue 
LDs  (Figure  26A;  solid  line)  across  the  -502  to  -42  msec  prestimulus  interval  shown  in  Figure  26A.  The  width  of  a  line 
shows  the  significance  of  the  covariance  between  two  electrodes.  Violet  lines  represent  positive  covariances,  while  blue  lines 
represent  negative  covariances.  The  midline  central  electrode,  overlying  the  motor  and  premotor  cortices,  significantly 
covaries  with  left  mid-parietal  and  right  poetero-parietal  electrodes,  and  left  and  right  superior  antero-parietal,  electrodes. 
This  suggests  that  incipient  fatigue  selectively  affected  cortical  areas  responsible  for  preparing  for  a  response,  and 
maintaining  the  two  visually  presented  numbers  in  workiug  memory. 
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The  N125  LD  peak  in  the  response  trial  set  grand-average  was  highly  localized  to 
1  the  midline  pexietal  site  due  to  the  Laplacian  transformation.  Its  amplitude  declined 

l  slightly  (3.1  to  2.9  /iV/  cm2  )  from  the  Alert  to  Incipient  Fatigue  conditions,  and 

['  then  increased  slightly  in  the  Full  Fatigue  condition  (3.3  fiV/  enr  ). 

|  For  the  interval  spanning  the  stimulus-locked  N125  peak,  none  of  the  ERCs  that 

were  computed  from  the  Alert-minus-Incipient-Fatigue  subtraction  LDs  (Fig.  27A) 
were  significant  (Fig.  27B).  For  the  Incipient-minus-Full-Fatigue  subtraction,  there 
was  only  one  significant  ERC,  between  the  left  and  right  poetero-parietal  sites  (not 
f  shown).  There  were  too  few  trials  in  the  no-response  trial  set  to  see  the  Nl  peak 

against  the  noise. 


Flgurt  37 A 

Alert  (562  trials) .  (dished  line),  Incipient  Fatigue  (517  trials)  (dotted  line),  and  Alert- minus-!ncipient-Fatigue  grand-average 
LDs  (solid  line)  of  the  midline  mid-parietal  (K*)  electrode,  showing  the  N125  peak  following  presentation  of  the  visual 
stimulus.  The  amplitude  of  the  N125  did  not  differ  between  the  Alert  and  Incipient  Fatigue  conditions. 


The  amplitude  of  the  P380  in  the  no-response  trial  set  significantly  decreased 
across  alertness  conditions  (F=7.3,  df=2,12,  p<.01).  Scheffe  post-hoc  tests  revealed 
that  the  amplitude  of  P380  was  significantly  smaller  in  the  Full  Fatigue  average 
than  in  the  Alert  average  (F=7.3,  df=2,12,  p<.01).  The  decrease  in  P380 
amplitude  from  the  Alert  to  the  Incipient  Fatigue  condition  did  not  reach  statistical 
significance  (Fig.  28 A). 
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Figure  MA 

Alert  (243  trials)  (dashed  line),  Incipient  Fatigue  (250  triali)  (dotted  line),  and  Vlert- minus- Incipient-Fatigue  grand-average 
LDj  (solid  line)  of  the  midline  antero-central  (aCi)  electrode  for  the  response  Inhibition  ('uo-pre*s')  trials.  The  decrease  in 
amplitude  of  the  P380  peak  from  Alert  to  Incipient-Fatigue  conditions  was  not  significant. 


Significant  ERCs  from  the  P380  Alert-minus- Incipient-Fatigue  subtraction,  in  the 
no-response  trial  set,  were  observed  between  the  midline  antero-central,  right 
antero-parietal  and  right  po6tero-parietal  sites  (Fig.  28B).  The  values  of  the  three 
significant  ERCs  were:  -19  (midline  antero-central  to  right  inferior  antero-parietal) , 
19  (midline  antero-central  to  right  postero-parietal),  and  -15  (right  postero-parietal 
to  right  inferior  antero-parietal).  The  mean  and  standard  deviation  of  the  ERC 
noise  distribution  were  0  and  2.8,  respectively.  No  significant  ERCs  were  found 
for  the  Incipient-minus-Full-Fatigue  P380  LD. 


Pattern  classification  procedures  were  used  to  measure  the  trial-by-trial 
discrkninability  of  Alert  from  Incipient  Fatigue  prestimulus  data,  and  to  provide  a 
summary  index  of  differences  between  subjects.  A  two-layered  adaptive  "neural 
network*  algorithm  was  used  for  the  analysis.  Single-trial  ERCs,  computed  for 
channel  pairs  of  the  ERC  patterns  from  both  subtracted  and  unsubtracted  grand- 
average  ERPs,  were  considered  as  possible  variables.  The  ratio  of  observations 
(trials)  to  variables  exceeded  10:1  in  each  analysis.  In  each  case,  five  analyses  were 
performed,  each-  using  four-fifths  of  the  data  for  training  and  the  remaining  fifth  for 
testing  significance  of  the  average  test  set  classification  was  assessed  by  reference  to 
the  bimanual  distribution. 

The  average  test-set  discrimination  of  Incipient  Fatigue  from  Alert  trials  was  58% 
(p<0.001),  where  five  separate  equations  were  formed  on  four-fifths  of  the  trials  and 
tested  on  the  remaining  fifth.  Discrimination  was  above  57%  for  three  subjects,  but 
was  49%  (chance)  for  the  other  two.  In  a  second  analysis  using  just  the  trials  from 
the  latter  two  subjects,  average  test-set  discrimination  was  61%  (p<0.001),  implying 
that  ERC  differences  between  Alert  and  Incipient  Fatigue  conditions  were 
distributed  differently  from  those  of  the  first  three  subjects.  A  third  analysis  using 
just  the  trials  from  the  first  three  subjects  (representing  75%  of  the  tdtal  trials), 
yielded  an  average  test-set  discrimination  of  62%  (pCO.OOl).  Finally,  a  fourth 
analysis  was  made  using  only  trials  from  the  one  subject  with  the  best  classification 
in  the  five-subject  and  three-subject  analyses.  The  average  test-set  discrimination  of 
Incipient  Fatigue  from  Alert  triaJs  was  81%  (p<0.001). 

5.  Discussion 

We  have  reported  preliminary  results  of  a  study  that  used  state-of-the-art 
technology  to  document  neurophysiological  activity  associated  with  different  aspects 
of  perceptuomotor  and  cognitive  function  in  five  U.S.  Air  Force  test  pilots  during 
three  stages  of  alertness:  full  alert,  incipient  fatigue  and  full  fatigue.  Over  one 
billion  bytes  of  data  were  collected.  We  have  summarized  the  results  from  the 
high-load  visuomotor  memory  task  (VMMT)  for  the  different  alertness  conditions, 
and  have  discussed  the  results  of  event-related  covariance  (ERC)  analysis  of  LDs 
from  event-related  potentials  obtained  by  subtracting  the  grand-average  5-subject 
LDs  from  the  different  alertness  states  for  the  stimulus-registered  event.  We  will 
present  the  results  cf  the  prestimulus-,  response-  and  feedback-registered  events  of 
the  VMMT,  Follow-Up  Rested  Session,  and  the  Auditory  Monitoring  and  Critical 
and  Subcritical  Tracking  Tasks  in  separate  reports. 

These  first  results  are  quite  exciting  in  suggesting  how  fatigue  may  selectively 
affect  areas  of  the  brain  implicated  in  higher  cognitive  functions.  The  N125  LD  peak 
and  corresponding  ERC  pattern  during  the  first  200  msec  of  the  visual  stimulus 
processing  were  relatively  unaffected.  By  contrast,  the  later  response-inhibition 
ERC  patterns,  from  about  280  to  470  msec  poststimulus  (and  to  some  extent  the 
P380  peaJc  amplitude),  clearly  showed  neurophysiological  changes  consequent  to 
incipient  fatigue.  Consistent  with  results  of  a  previous  study  in  our  laboratory  of 
visuomotor  performance  (39],  the  midline  antero-central  focus  of  the  P380  no¬ 
response  trial  ERC  patterns  may  reflect  changes  in  neural  systems  responsible  for 
motor  inhibition.  This  electrode  overlies  supplementary  motor  and  premotor 
cortices  involved  in  the  highest  level  of  control  of  voluntary  movements  (50,64 j. 


The  highly  specific  temporal  (LD  peaks)  and  spatial  (ERC  patterns)  changes 
associated  with  time-on-task  suggest  that  the  patterns  are  due  to  localized  changes 
in  areas  of  association  cortex,  and  not  to  anatomically  diffuse  changes  associated 
with  decreased  overall  arousal. 

We  must  also  emphasize  that  while  the  ERG  patterns  reported  here  suggest 
fatigue-related  degradation  of  functional  coordination  between  immediately 
underlying  cortical  regions,  the  actual  neural  sources  of  the  ERG  patterns  are,  in 
fact,  not  yet  known.  Determining  the  distributed  source  network  suggested  by  the 
scalp  ERG  patterns  is  the  major  focus  of  our  current  technical  efforts. 

The  present  results,  analyses  of  the  remaining  data,  and  source  localization 
studies  will  all  elucidate  the  differential  effects  of  incipient  fatigue  on  areas  of  the 
brain  implicated  in  higher  cognitive  functions.  When  these  analyses  are  completed, 
it  may  be  possible  to  specify  studies  leading  to  the  design  of  on-line  devices  for 
predicting  "transient  cognitive  lapses*  and  performance  decrements  resulting  from 
operational  fatigue.  The  same  technology  could  be  applied  to  predict  acceleration- 
induced  loss  of  consciousness  episodes  in  high  performance  aircraft. 
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